{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "75E57D15ECA94D9780DE3DCAFC461E9B",
    "jupyter": {},
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "# Python机器学习（第15期）第8课书面作业\n",
    "学号：113727  \n",
    "**作业内容：**  \n",
    "数据集（链接：https://pan.baidu.com/s/1pLzbTtp 密码：e3sn）是关于贷款违约的数据集，目标变量为Disbursed（1为违约，0为未违约）。对数据集做好基本的处理后，将数据集随机划分为训练集和测试集（划分比例8:2）。\n",
    "1. 对训练集数据构建XGBoost模型，并计算测试集的准确率和AUC值。可以尝试调节参数，感受不同参数对模型的影响。  \n",
    "2. 对训练数据使用SVM模型构建一个分类模型，理解SVM模型有哪些可以调整的超参数？并分别使用随机搜索，网格搜索，贝叶斯优化等方法对模型进行超参数调优，使得模型在测试集上准确率尽量高（最好达到95%或以上）。比较不同搜索方法的性能和结果。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "1C556B60F3C44CDE8C595666E0B5D3F3",
    "jupyter": {},
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## 数据预处理"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "_cell_guid": "b1076dfc-b9ad-4769-8c92-a6c4dae69d19",
    "_uuid": "8f2839f25d086af736a60e9eeb907d3b93b6e0e5",
    "execution": {
     "iopub.execute_input": "2021-09-04T08:41:18.758019Z",
     "iopub.status.busy": "2021-09-04T08:41:18.757555Z",
     "iopub.status.idle": "2021-09-04T08:41:18.770149Z",
     "shell.execute_reply": "2021-09-04T08:41:18.768820Z",
     "shell.execute_reply.started": "2021-09-04T08:41:18.757974Z"
    },
    "id": "71B333B2F31C42328B00F4787D9A85B2",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "import numpy as np # linear algebra\n",
    "import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-04T08:41:36.250072Z",
     "iopub.status.busy": "2021-09-04T08:41:36.249645Z",
     "iopub.status.idle": "2021-09-04T08:42:26.416997Z",
     "shell.execute_reply": "2021-09-04T08:42:26.415932Z",
     "shell.execute_reply.started": "2021-09-04T08:41:36.250020Z"
    },
    "id": "19A9C95CA72A4402BB3022E740BFEB28",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "df_train=pd.read_excel('train.xlsx')\n",
    "#df_train=pd.read_csv('/kaggle/input/loandisbursed/train.csv',low_memory=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-04T08:42:26.418780Z",
     "iopub.status.busy": "2021-09-04T08:42:26.418441Z",
     "iopub.status.idle": "2021-09-04T08:42:26.538954Z",
     "shell.execute_reply": "2021-09-04T08:42:26.538178Z",
     "shell.execute_reply.started": "2021-09-04T08:42:26.418748Z"
    },
    "id": "15BEDFCAC3A24E968EA2A810EEEE013C",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ID</th>\n",
       "      <th>Gender</th>\n",
       "      <th>City</th>\n",
       "      <th>Monthly_Income</th>\n",
       "      <th>DOB</th>\n",
       "      <th>Lead_Creation_Date</th>\n",
       "      <th>Loan_Amount_Applied</th>\n",
       "      <th>Loan_Tenure_Applied</th>\n",
       "      <th>Existing_EMI</th>\n",
       "      <th>Employer_Name</th>\n",
       "      <th>...</th>\n",
       "      <th>Interest_Rate</th>\n",
       "      <th>Processing_Fee</th>\n",
       "      <th>EMI_Loan_Submitted</th>\n",
       "      <th>Filled_Form</th>\n",
       "      <th>Device_Type</th>\n",
       "      <th>Var2</th>\n",
       "      <th>Source</th>\n",
       "      <th>Var4</th>\n",
       "      <th>LoggedIn</th>\n",
       "      <th>Disbursed</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>ID000002C20</td>\n",
       "      <td>Female</td>\n",
       "      <td>Delhi</td>\n",
       "      <td>20000</td>\n",
       "      <td>1978-05-23</td>\n",
       "      <td>2015-05-15</td>\n",
       "      <td>300000.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>CYBOSOL</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>N</td>\n",
       "      <td>Web-browser</td>\n",
       "      <td>G</td>\n",
       "      <td>S122</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ID000004E40</td>\n",
       "      <td>Male</td>\n",
       "      <td>Mumbai</td>\n",
       "      <td>35000</td>\n",
       "      <td>1985-10-07</td>\n",
       "      <td>2015-05-04</td>\n",
       "      <td>200000.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>TATA CONSULTANCY SERVICES LTD (TCS)</td>\n",
       "      <td>...</td>\n",
       "      <td>13.25</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6762.9</td>\n",
       "      <td>N</td>\n",
       "      <td>Web-browser</td>\n",
       "      <td>G</td>\n",
       "      <td>S122</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>ID000007H20</td>\n",
       "      <td>Male</td>\n",
       "      <td>Panchkula</td>\n",
       "      <td>22500</td>\n",
       "      <td>1981-10-10</td>\n",
       "      <td>2015-05-19</td>\n",
       "      <td>600000.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>ALCHEMIST HOSPITALS LTD</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>N</td>\n",
       "      <td>Web-browser</td>\n",
       "      <td>B</td>\n",
       "      <td>S143</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>ID000008I30</td>\n",
       "      <td>Male</td>\n",
       "      <td>Saharsa</td>\n",
       "      <td>35000</td>\n",
       "      <td>1987-11-30</td>\n",
       "      <td>2015-05-09</td>\n",
       "      <td>1000000.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>BIHAR GOVERNMENT</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>N</td>\n",
       "      <td>Web-browser</td>\n",
       "      <td>B</td>\n",
       "      <td>S143</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>ID000009J40</td>\n",
       "      <td>Male</td>\n",
       "      <td>Bengaluru</td>\n",
       "      <td>100000</td>\n",
       "      <td>1984-02-17</td>\n",
       "      <td>2015-05-20</td>\n",
       "      <td>500000.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>25000.0</td>\n",
       "      <td>GLOBAL EDGE SOFTWARE</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>N</td>\n",
       "      <td>Web-browser</td>\n",
       "      <td>B</td>\n",
       "      <td>S134</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 26 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "            ID  Gender       City  Monthly_Income        DOB  \\\n",
       "0  ID000002C20  Female      Delhi           20000 1978-05-23   \n",
       "1  ID000004E40    Male     Mumbai           35000 1985-10-07   \n",
       "2  ID000007H20    Male  Panchkula           22500 1981-10-10   \n",
       "3  ID000008I30    Male    Saharsa           35000 1987-11-30   \n",
       "4  ID000009J40    Male  Bengaluru          100000 1984-02-17   \n",
       "\n",
       "  Lead_Creation_Date  Loan_Amount_Applied  Loan_Tenure_Applied  Existing_EMI  \\\n",
       "0         2015-05-15             300000.0                  5.0           0.0   \n",
       "1         2015-05-04             200000.0                  2.0           0.0   \n",
       "2         2015-05-19             600000.0                  4.0           0.0   \n",
       "3         2015-05-09            1000000.0                  5.0           0.0   \n",
       "4         2015-05-20             500000.0                  2.0       25000.0   \n",
       "\n",
       "                         Employer_Name  ... Interest_Rate Processing_Fee  \\\n",
       "0                              CYBOSOL  ...           NaN            NaN   \n",
       "1  TATA CONSULTANCY SERVICES LTD (TCS)  ...         13.25            NaN   \n",
       "2              ALCHEMIST HOSPITALS LTD  ...           NaN            NaN   \n",
       "3                     BIHAR GOVERNMENT  ...           NaN            NaN   \n",
       "4                 GLOBAL EDGE SOFTWARE  ...           NaN            NaN   \n",
       "\n",
       "  EMI_Loan_Submitted Filled_Form  Device_Type  Var2  Source  Var4 LoggedIn  \\\n",
       "0                NaN           N  Web-browser     G    S122     1        0   \n",
       "1             6762.9           N  Web-browser     G    S122     3        0   \n",
       "2                NaN           N  Web-browser     B    S143     1        0   \n",
       "3                NaN           N  Web-browser     B    S143     3        0   \n",
       "4                NaN           N  Web-browser     B    S134     3        1   \n",
       "\n",
       "  Disbursed  \n",
       "0       0.0  \n",
       "1       0.0  \n",
       "2       0.0  \n",
       "3       0.0  \n",
       "4       0.0  \n",
       "\n",
       "[5 rows x 26 columns]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_train.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "BD12255E6C124A7AB24C75765A07E145",
    "jupyter": {},
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "数据集共26列（包含1个目标变量，25个因变量），共87020个样本。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-04T08:42:26.540693Z",
     "iopub.status.busy": "2021-09-04T08:42:26.540205Z",
     "iopub.status.idle": "2021-09-04T08:42:26.675530Z",
     "shell.execute_reply": "2021-09-04T08:42:26.674802Z",
     "shell.execute_reply.started": "2021-09-04T08:42:26.540656Z"
    },
    "id": "0EFA7298DE964FB9882D22C8127707F3",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 87020 entries, 0 to 87019\n",
      "Data columns (total 26 columns):\n",
      " #   Column                 Non-Null Count  Dtype         \n",
      "---  ------                 --------------  -----         \n",
      " 0   ID                     87020 non-null  object        \n",
      " 1   Gender                 87020 non-null  object        \n",
      " 2   City                   86017 non-null  object        \n",
      " 3   Monthly_Income         87020 non-null  int64         \n",
      " 4   DOB                    87020 non-null  datetime64[ns]\n",
      " 5   Lead_Creation_Date     87020 non-null  datetime64[ns]\n",
      " 6   Loan_Amount_Applied    86949 non-null  float64       \n",
      " 7   Loan_Tenure_Applied    86949 non-null  float64       \n",
      " 8   Existing_EMI           86949 non-null  float64       \n",
      " 9   Employer_Name          86949 non-null  object        \n",
      " 10  Salary_Account         75256 non-null  object        \n",
      " 11  Mobile_Verified        87020 non-null  object        \n",
      " 12  Var5                   87020 non-null  object        \n",
      " 13  Var1                   87019 non-null  object        \n",
      " 14  Loan_Amount_Submitted  52407 non-null  float64       \n",
      " 15  Loan_Tenure_Submitted  52407 non-null  float64       \n",
      " 16  Interest_Rate          27726 non-null  float64       \n",
      " 17  Processing_Fee         27420 non-null  float64       \n",
      " 18  EMI_Loan_Submitted     27727 non-null  object        \n",
      " 19  Filled_Form            87020 non-null  object        \n",
      " 20  Device_Type            87020 non-null  object        \n",
      " 21  Var2                   87020 non-null  object        \n",
      " 22  Source                 87020 non-null  object        \n",
      " 23  Var4                   87020 non-null  int64         \n",
      " 24  LoggedIn               87020 non-null  int64         \n",
      " 25  Disbursed              87019 non-null  float64       \n",
      "dtypes: datetime64[ns](2), float64(8), int64(3), object(13)\n",
      "memory usage: 17.3+ MB\n"
     ]
    }
   ],
   "source": [
    "df_train.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "43AAD1C0E0C34DE4BBD1B88B6749768B",
    "jupyter": {},
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "* 可以看到City、Loan_Amount_Applied、Loan_Tenure_Applied、Existing_EMI、Employer_Name、Salary_Account、Var1、Loan_Amount_Submitted、Loan_Tenure_Submitted、Interest_rate、Processing_Fee、EMI_Loan_Submitted存在或多或少的缺失值。\n",
    "目标变量也存在一个缺失值。\n",
    "* 同时存在数值型变量与分类型变量。  \n",
    "\n",
    "对于数值型数据，考虑对缺失值采用均值填充处理。对于目标变量，众值填充。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-04T08:42:26.678169Z",
     "iopub.status.busy": "2021-09-04T08:42:26.677637Z",
     "iopub.status.idle": "2021-09-04T08:42:26.820905Z",
     "shell.execute_reply": "2021-09-04T08:42:26.820092Z",
     "shell.execute_reply.started": "2021-09-04T08:42:26.678136Z"
    },
    "id": "796D57D88EA1452A9E62081B2C1E8DD3",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 87020 entries, 0 to 87019\n",
      "Data columns (total 26 columns):\n",
      " #   Column                 Non-Null Count  Dtype         \n",
      "---  ------                 --------------  -----         \n",
      " 0   ID                     87020 non-null  object        \n",
      " 1   Gender                 87020 non-null  object        \n",
      " 2   City                   86017 non-null  object        \n",
      " 3   Monthly_Income         87020 non-null  int64         \n",
      " 4   DOB                    87020 non-null  datetime64[ns]\n",
      " 5   Lead_Creation_Date     87020 non-null  datetime64[ns]\n",
      " 6   Loan_Amount_Applied    87020 non-null  float64       \n",
      " 7   Loan_Tenure_Applied    87020 non-null  float64       \n",
      " 8   Existing_EMI           87020 non-null  float64       \n",
      " 9   Employer_Name          86949 non-null  object        \n",
      " 10  Salary_Account         75256 non-null  object        \n",
      " 11  Mobile_Verified        87020 non-null  object        \n",
      " 12  Var5                   87020 non-null  object        \n",
      " 13  Var1                   87019 non-null  object        \n",
      " 14  Loan_Amount_Submitted  87020 non-null  float64       \n",
      " 15  Loan_Tenure_Submitted  87020 non-null  float64       \n",
      " 16  Interest_Rate          87020 non-null  float64       \n",
      " 17  Processing_Fee         87020 non-null  float64       \n",
      " 18  EMI_Loan_Submitted     27727 non-null  object        \n",
      " 19  Filled_Form            87020 non-null  object        \n",
      " 20  Device_Type            87020 non-null  object        \n",
      " 21  Var2                   87020 non-null  object        \n",
      " 22  Source                 87020 non-null  object        \n",
      " 23  Var4                   87020 non-null  int64         \n",
      " 24  LoggedIn               87020 non-null  int64         \n",
      " 25  Disbursed              87020 non-null  float64       \n",
      "dtypes: datetime64[ns](2), float64(8), int64(3), object(13)\n",
      "memory usage: 17.3+ MB\n"
     ]
    }
   ],
   "source": [
    "numcols=df_train.select_dtypes(include=['int64','float64']).columns.tolist()\n",
    "numcols.remove('Disbursed')\n",
    "for a in numcols:\n",
    "    df_train[a]=df_train[a].fillna(df_train[a].mean())\n",
    "df_train['Disbursed']=df_train['Disbursed'].fillna(0)\n",
    "df_train.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-04T08:42:26.823423Z",
     "iopub.status.busy": "2021-09-04T08:42:26.823110Z",
     "iopub.status.idle": "2021-09-04T08:42:26.831067Z",
     "shell.execute_reply": "2021-09-04T08:42:26.829909Z",
     "shell.execute_reply.started": "2021-09-04T08:42:26.823393Z"
    },
    "id": "06E85B1729CF4F908F3F69FE82E493D2",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "def is_number(s):\n",
    "    try:\n",
    "        float(s)\n",
    "        return True\n",
    "    except ValueError:\n",
    "        pass\n",
    "    \n",
    "    try:\n",
    "        import unicodedata\n",
    "        unicodedata.numeric(s)\n",
    "        return True\n",
    "    except (TypeError, ValueError):\n",
    "        pass\n",
    "    return False"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0CB41B614D8E480B916070B1710F3B7B",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "很奇怪，'EMI_Loan_Submitted'应该是数值型数据，这里显示为Object类型，估计是存在一些样本中该变量有奇怪的取值。下面来查看一下。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-04T08:42:26.832691Z",
     "iopub.status.busy": "2021-09-04T08:42:26.832381Z",
     "iopub.status.idle": "2021-09-04T08:42:27.964579Z",
     "shell.execute_reply": "2021-09-04T08:42:27.963466Z",
     "shell.execute_reply.started": "2021-09-04T08:42:26.832656Z"
    },
    "id": "E6EE25D91A79444089026BABA1E0BD3A",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "62689 6762.9\n"
     ]
    }
   ],
   "source": [
    "for i in range(df_train['EMI_Loan_Submitted'].shape[0]):\n",
    "    if is_number(df_train['EMI_Loan_Submitted'].iloc[i])==False:\n",
    "        print(i,df_train['EMI_Loan_Submitted'].iloc[1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-04T08:42:27.966641Z",
     "iopub.status.busy": "2021-09-04T08:42:27.966194Z",
     "iopub.status.idle": "2021-09-04T08:42:27.974406Z",
     "shell.execute_reply": "2021-09-04T08:42:27.973219Z",
     "shell.execute_reply.started": "2021-09-04T08:42:27.966600Z"
    },
    "id": "683D93DDE075436D8D08CBAB371C54C5",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'N'"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_train['EMI_Loan_Submitted'].loc[62689]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "4E70EE00A67E4F4290990AB645718C40",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "第62689样本的'EMI_Loan_Submitted'取值是比较奇怪，下面用数值重新写一下。再转换为float类型。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-04T08:42:27.976341Z",
     "iopub.status.busy": "2021-09-04T08:42:27.975957Z",
     "iopub.status.idle": "2021-09-04T08:42:28.082327Z",
     "shell.execute_reply": "2021-09-04T08:42:28.080997Z",
     "shell.execute_reply.started": "2021-09-04T08:42:27.976305Z"
    },
    "id": "438C443E5C73489387B07ED5082BE699",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "df_train.at[62689,'EMI_Loan_Submitted']=6762.9\n",
    "df_train['EMI_Loan_Submitted']=df_train['EMI_Loan_Submitted'].fillna(0)\n",
    "df_train['EMI_Loan_Submitted']=df_train['EMI_Loan_Submitted'].astype('float')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "8C92FA46950F4A849D817FE18B25A0E1",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "下面来看一下分类型变量。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-04T08:42:28.084098Z",
     "iopub.status.busy": "2021-09-04T08:42:28.083780Z",
     "iopub.status.idle": "2021-09-04T08:42:28.297515Z",
     "shell.execute_reply": "2021-09-04T08:42:28.295985Z",
     "shell.execute_reply.started": "2021-09-04T08:42:28.084065Z"
    },
    "id": "4B4A946612AC4D3588319BFAD0CD2736",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>City</th>\n",
       "      <th>Employer_Name</th>\n",
       "      <th>Salary_Account</th>\n",
       "      <th>Var1</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>86017</td>\n",
       "      <td>86949</td>\n",
       "      <td>75256</td>\n",
       "      <td>87019</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>unique</th>\n",
       "      <td>697</td>\n",
       "      <td>43567</td>\n",
       "      <td>58</td>\n",
       "      <td>19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>top</th>\n",
       "      <td>Delhi</td>\n",
       "      <td>0</td>\n",
       "      <td>HDFC Bank</td>\n",
       "      <td>HBXX</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>freq</th>\n",
       "      <td>12527</td>\n",
       "      <td>4914</td>\n",
       "      <td>17695</td>\n",
       "      <td>59293</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         City  Employer_Name Salary_Account   Var1\n",
       "count   86017          86949          75256  87019\n",
       "unique    697          43567             58     19\n",
       "top     Delhi              0      HDFC Bank   HBXX\n",
       "freq    12527           4914          17695  59293"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nulobjcols=['City','Employer_Name','Salary_Account','Var1']\n",
    "df_train[nulobjcols].describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "96EB70CC15904B119D6CF74A7303D19F",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "用“unknow”来填充这些缺失值。上面处理的数据集中没有缺失值了。保存为csv格式。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-04T08:42:28.299448Z",
     "iopub.status.busy": "2021-09-04T08:42:28.299131Z",
     "iopub.status.idle": "2021-09-04T08:42:31.343116Z",
     "shell.execute_reply": "2021-09-04T08:42:31.342319Z",
     "shell.execute_reply.started": "2021-09-04T08:42:28.299416Z"
    },
    "id": "99D67217F79E44DB8774A03A1B6DAB5C",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "for a in nulobjcols:\n",
    "    df_train[a]=df_train[a].fillna('unknown')\n",
    "df_train['Disbursed']=df_train['Disbursed'].astype('int')\n",
    "df_train.to_csv('train.csv',index=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-04T08:42:31.345431Z",
     "iopub.status.busy": "2021-09-04T08:42:31.344723Z",
     "iopub.status.idle": "2021-09-04T08:42:31.943788Z",
     "shell.execute_reply": "2021-09-04T08:42:31.942359Z",
     "shell.execute_reply.started": "2021-09-04T08:42:31.345385Z"
    },
    "id": "29AE4162798C404F8235A6B061988E9C",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "df_train=pd.read_csv('train.csv',low_memory=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-04T08:42:31.947758Z",
     "iopub.status.busy": "2021-09-04T08:42:31.947431Z",
     "iopub.status.idle": "2021-09-04T08:42:31.978852Z",
     "shell.execute_reply": "2021-09-04T08:42:31.978170Z",
     "shell.execute_reply.started": "2021-09-04T08:42:31.947728Z"
    },
    "id": "293002574F7B4DCF8C4B07ED80020802",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ID</th>\n",
       "      <th>Gender</th>\n",
       "      <th>City</th>\n",
       "      <th>Monthly_Income</th>\n",
       "      <th>DOB</th>\n",
       "      <th>Lead_Creation_Date</th>\n",
       "      <th>Loan_Amount_Applied</th>\n",
       "      <th>Loan_Tenure_Applied</th>\n",
       "      <th>Existing_EMI</th>\n",
       "      <th>Employer_Name</th>\n",
       "      <th>...</th>\n",
       "      <th>Interest_Rate</th>\n",
       "      <th>Processing_Fee</th>\n",
       "      <th>EMI_Loan_Submitted</th>\n",
       "      <th>Filled_Form</th>\n",
       "      <th>Device_Type</th>\n",
       "      <th>Var2</th>\n",
       "      <th>Source</th>\n",
       "      <th>Var4</th>\n",
       "      <th>LoggedIn</th>\n",
       "      <th>Disbursed</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>ID000002C20</td>\n",
       "      <td>Female</td>\n",
       "      <td>Delhi</td>\n",
       "      <td>20000</td>\n",
       "      <td>1978-05-23</td>\n",
       "      <td>2015-05-15</td>\n",
       "      <td>300000.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>CYBOSOL</td>\n",
       "      <td>...</td>\n",
       "      <td>19.197474</td>\n",
       "      <td>5131.150839</td>\n",
       "      <td>0.0</td>\n",
       "      <td>N</td>\n",
       "      <td>Web-browser</td>\n",
       "      <td>G</td>\n",
       "      <td>S122</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ID000004E40</td>\n",
       "      <td>Male</td>\n",
       "      <td>Mumbai</td>\n",
       "      <td>35000</td>\n",
       "      <td>1985-10-07</td>\n",
       "      <td>2015-05-04</td>\n",
       "      <td>200000.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>TATA CONSULTANCY SERVICES LTD (TCS)</td>\n",
       "      <td>...</td>\n",
       "      <td>13.250000</td>\n",
       "      <td>5131.150839</td>\n",
       "      <td>6762.9</td>\n",
       "      <td>N</td>\n",
       "      <td>Web-browser</td>\n",
       "      <td>G</td>\n",
       "      <td>S122</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>ID000007H20</td>\n",
       "      <td>Male</td>\n",
       "      <td>Panchkula</td>\n",
       "      <td>22500</td>\n",
       "      <td>1981-10-10</td>\n",
       "      <td>2015-05-19</td>\n",
       "      <td>600000.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>ALCHEMIST HOSPITALS LTD</td>\n",
       "      <td>...</td>\n",
       "      <td>19.197474</td>\n",
       "      <td>5131.150839</td>\n",
       "      <td>0.0</td>\n",
       "      <td>N</td>\n",
       "      <td>Web-browser</td>\n",
       "      <td>B</td>\n",
       "      <td>S143</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>ID000008I30</td>\n",
       "      <td>Male</td>\n",
       "      <td>Saharsa</td>\n",
       "      <td>35000</td>\n",
       "      <td>1987-11-30</td>\n",
       "      <td>2015-05-09</td>\n",
       "      <td>1000000.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>BIHAR GOVERNMENT</td>\n",
       "      <td>...</td>\n",
       "      <td>19.197474</td>\n",
       "      <td>5131.150839</td>\n",
       "      <td>0.0</td>\n",
       "      <td>N</td>\n",
       "      <td>Web-browser</td>\n",
       "      <td>B</td>\n",
       "      <td>S143</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>ID000009J40</td>\n",
       "      <td>Male</td>\n",
       "      <td>Bengaluru</td>\n",
       "      <td>100000</td>\n",
       "      <td>1984-02-17</td>\n",
       "      <td>2015-05-20</td>\n",
       "      <td>500000.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>25000.0</td>\n",
       "      <td>GLOBAL EDGE SOFTWARE</td>\n",
       "      <td>...</td>\n",
       "      <td>19.197474</td>\n",
       "      <td>5131.150839</td>\n",
       "      <td>0.0</td>\n",
       "      <td>N</td>\n",
       "      <td>Web-browser</td>\n",
       "      <td>B</td>\n",
       "      <td>S134</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 26 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "            ID  Gender       City  Monthly_Income         DOB  \\\n",
       "0  ID000002C20  Female      Delhi           20000  1978-05-23   \n",
       "1  ID000004E40    Male     Mumbai           35000  1985-10-07   \n",
       "2  ID000007H20    Male  Panchkula           22500  1981-10-10   \n",
       "3  ID000008I30    Male    Saharsa           35000  1987-11-30   \n",
       "4  ID000009J40    Male  Bengaluru          100000  1984-02-17   \n",
       "\n",
       "  Lead_Creation_Date  Loan_Amount_Applied  Loan_Tenure_Applied  Existing_EMI  \\\n",
       "0         2015-05-15             300000.0                  5.0           0.0   \n",
       "1         2015-05-04             200000.0                  2.0           0.0   \n",
       "2         2015-05-19             600000.0                  4.0           0.0   \n",
       "3         2015-05-09            1000000.0                  5.0           0.0   \n",
       "4         2015-05-20             500000.0                  2.0       25000.0   \n",
       "\n",
       "                         Employer_Name  ... Interest_Rate Processing_Fee  \\\n",
       "0                              CYBOSOL  ...     19.197474    5131.150839   \n",
       "1  TATA CONSULTANCY SERVICES LTD (TCS)  ...     13.250000    5131.150839   \n",
       "2              ALCHEMIST HOSPITALS LTD  ...     19.197474    5131.150839   \n",
       "3                     BIHAR GOVERNMENT  ...     19.197474    5131.150839   \n",
       "4                 GLOBAL EDGE SOFTWARE  ...     19.197474    5131.150839   \n",
       "\n",
       "  EMI_Loan_Submitted Filled_Form  Device_Type  Var2  Source  Var4  LoggedIn  \\\n",
       "0                0.0           N  Web-browser     G    S122     1         0   \n",
       "1             6762.9           N  Web-browser     G    S122     3         0   \n",
       "2                0.0           N  Web-browser     B    S143     1         0   \n",
       "3                0.0           N  Web-browser     B    S143     3         0   \n",
       "4                0.0           N  Web-browser     B    S134     3         1   \n",
       "\n",
       "  Disbursed  \n",
       "0         0  \n",
       "1         0  \n",
       "2         0  \n",
       "3         0  \n",
       "4         0  \n",
       "\n",
       "[5 rows x 26 columns]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-04T08:42:31.980234Z",
     "iopub.status.busy": "2021-09-04T08:42:31.979835Z",
     "iopub.status.idle": "2021-09-04T08:42:32.126550Z",
     "shell.execute_reply": "2021-09-04T08:42:32.125314Z",
     "shell.execute_reply.started": "2021-09-04T08:42:31.980206Z"
    },
    "id": "473595563C604EF09C15607F707A2862",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 87020 entries, 0 to 87019\n",
      "Data columns (total 26 columns):\n",
      " #   Column                 Non-Null Count  Dtype  \n",
      "---  ------                 --------------  -----  \n",
      " 0   ID                     87020 non-null  object \n",
      " 1   Gender                 87020 non-null  object \n",
      " 2   City                   87020 non-null  object \n",
      " 3   Monthly_Income         87020 non-null  int64  \n",
      " 4   DOB                    87020 non-null  object \n",
      " 5   Lead_Creation_Date     87020 non-null  object \n",
      " 6   Loan_Amount_Applied    87020 non-null  float64\n",
      " 7   Loan_Tenure_Applied    87020 non-null  float64\n",
      " 8   Existing_EMI           87020 non-null  float64\n",
      " 9   Employer_Name          87020 non-null  object \n",
      " 10  Salary_Account         87020 non-null  object \n",
      " 11  Mobile_Verified        87020 non-null  object \n",
      " 12  Var5                   87020 non-null  object \n",
      " 13  Var1                   87020 non-null  object \n",
      " 14  Loan_Amount_Submitted  87020 non-null  float64\n",
      " 15  Loan_Tenure_Submitted  87020 non-null  float64\n",
      " 16  Interest_Rate          87020 non-null  float64\n",
      " 17  Processing_Fee         87020 non-null  float64\n",
      " 18  EMI_Loan_Submitted     87020 non-null  float64\n",
      " 19  Filled_Form            87020 non-null  object \n",
      " 20  Device_Type            87020 non-null  object \n",
      " 21  Var2                   87020 non-null  object \n",
      " 22  Source                 87020 non-null  object \n",
      " 23  Var4                   87020 non-null  int64  \n",
      " 24  LoggedIn               87020 non-null  int64  \n",
      " 25  Disbursed              87020 non-null  int64  \n",
      "dtypes: float64(8), int64(4), object(14)\n",
      "memory usage: 17.3+ MB\n"
     ]
    }
   ],
   "source": [
    "df_train.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-04T08:42:32.128663Z",
     "iopub.status.busy": "2021-09-04T08:42:32.128034Z",
     "iopub.status.idle": "2021-09-04T08:42:32.687015Z",
     "shell.execute_reply": "2021-09-04T08:42:32.686005Z",
     "shell.execute_reply.started": "2021-09-04T08:42:32.128615Z"
    },
    "id": "E92CB9A245C844DE870BC352A840A75E",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ID</th>\n",
       "      <th>Gender</th>\n",
       "      <th>City</th>\n",
       "      <th>DOB</th>\n",
       "      <th>Lead_Creation_Date</th>\n",
       "      <th>Employer_Name</th>\n",
       "      <th>Salary_Account</th>\n",
       "      <th>Mobile_Verified</th>\n",
       "      <th>Var5</th>\n",
       "      <th>Var1</th>\n",
       "      <th>Filled_Form</th>\n",
       "      <th>Device_Type</th>\n",
       "      <th>Var2</th>\n",
       "      <th>Source</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>87020</td>\n",
       "      <td>87020</td>\n",
       "      <td>87020</td>\n",
       "      <td>87020</td>\n",
       "      <td>87020</td>\n",
       "      <td>87020</td>\n",
       "      <td>87020</td>\n",
       "      <td>87020</td>\n",
       "      <td>87020</td>\n",
       "      <td>87020</td>\n",
       "      <td>87020</td>\n",
       "      <td>87020</td>\n",
       "      <td>87020</td>\n",
       "      <td>87020</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>unique</th>\n",
       "      <td>87020</td>\n",
       "      <td>2</td>\n",
       "      <td>698</td>\n",
       "      <td>11345</td>\n",
       "      <td>92</td>\n",
       "      <td>43568</td>\n",
       "      <td>59</td>\n",
       "      <td>3</td>\n",
       "      <td>20</td>\n",
       "      <td>20</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>8</td>\n",
       "      <td>31</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>top</th>\n",
       "      <td>ID054051X10</td>\n",
       "      <td>Male</td>\n",
       "      <td>Delhi</td>\n",
       "      <td>1980-11-11</td>\n",
       "      <td>2015-07-03</td>\n",
       "      <td>0</td>\n",
       "      <td>HDFC Bank</td>\n",
       "      <td>Y</td>\n",
       "      <td>0</td>\n",
       "      <td>HBXX</td>\n",
       "      <td>N</td>\n",
       "      <td>Web-browser</td>\n",
       "      <td>B</td>\n",
       "      <td>S122</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>freq</th>\n",
       "      <td>1</td>\n",
       "      <td>49848</td>\n",
       "      <td>12527</td>\n",
       "      <td>306</td>\n",
       "      <td>2315</td>\n",
       "      <td>4914</td>\n",
       "      <td>17695</td>\n",
       "      <td>56481</td>\n",
       "      <td>29086</td>\n",
       "      <td>59293</td>\n",
       "      <td>67529</td>\n",
       "      <td>64316</td>\n",
       "      <td>37280</td>\n",
       "      <td>38566</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 ID Gender   City         DOB Lead_Creation_Date  \\\n",
       "count         87020  87020  87020       87020              87020   \n",
       "unique        87020      2    698       11345                 92   \n",
       "top     ID054051X10   Male  Delhi  1980-11-11         2015-07-03   \n",
       "freq              1  49848  12527         306               2315   \n",
       "\n",
       "       Employer_Name Salary_Account Mobile_Verified   Var5   Var1 Filled_Form  \\\n",
       "count          87020          87020           87020  87020  87020       87020   \n",
       "unique         43568             59               3     20     20           3   \n",
       "top                0      HDFC Bank               Y      0   HBXX           N   \n",
       "freq            4914          17695           56481  29086  59293       67529   \n",
       "\n",
       "        Device_Type   Var2 Source  \n",
       "count         87020  87020  87020  \n",
       "unique            3      8     31  \n",
       "top     Web-browser      B   S122  \n",
       "freq          64316  37280  38566  "
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_train.select_dtypes(include=['O']).describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0F36A6F30B144A398A4F1BE5CB46D28B",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "下面对分类型变量进行数值化转换。  \n",
    "* Gender\\Mobile_Verified\\Filled_Form\\Device_Type：采用one-hot编码\n",
    "* City\\Salary_Account\\Var5\\Var1\\Var2\\Source：采用Target编码\n",
    "* ID\\Employer_Name：不作为训练数据\n",
    "* DOB\\Lead_Creation_Date: 转换为年/月/日后用Target编码"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-04T08:42:32.688591Z",
     "iopub.status.busy": "2021-09-04T08:42:32.688299Z",
     "iopub.status.idle": "2021-09-04T08:42:32.837192Z",
     "shell.execute_reply": "2021-09-04T08:42:32.835972Z",
     "shell.execute_reply.started": "2021-09-04T08:42:32.688557Z"
    },
    "id": "46F5C45B73874210BBB90F92C8D778DB",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "df_raw=df_train.drop(columns=['ID','Employer_Name'])\n",
    "df_raw['DOB_dt']=pd.to_datetime(df_raw['DOB'])\n",
    "df_raw['LCD_dt']=pd.to_datetime(df_raw['Lead_Creation_Date'])\n",
    "df_raw['DOB_year']=df_raw['DOB_dt'].dt.year\n",
    "df_raw['DOB_month']=df_raw['DOB_dt'].dt.month\n",
    "df_raw['DOB_day']=df_raw['DOB_dt'].dt.day\n",
    "df_raw['LCD_year']=df_raw['LCD_dt'].dt.year\n",
    "df_raw['LCD_month']=df_raw['LCD_dt'].dt.month\n",
    "df_raw['LCD_day']=df_raw['LCD_dt'].dt.day\n",
    "df_raw=df_raw.drop(columns=['DOB','Lead_Creation_Date','DOB_dt','LCD_dt'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-04T08:42:32.841092Z",
     "iopub.status.busy": "2021-09-04T08:42:32.840716Z",
     "iopub.status.idle": "2021-09-04T08:42:32.911287Z",
     "shell.execute_reply": "2021-09-04T08:42:32.910097Z",
     "shell.execute_reply.started": "2021-09-04T08:42:32.841038Z"
    },
    "id": "358A56427ABD40F48226C58259F66AC2",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Gender</th>\n",
       "      <th>City</th>\n",
       "      <th>Monthly_Income</th>\n",
       "      <th>Loan_Amount_Applied</th>\n",
       "      <th>Loan_Tenure_Applied</th>\n",
       "      <th>Existing_EMI</th>\n",
       "      <th>Salary_Account</th>\n",
       "      <th>Mobile_Verified</th>\n",
       "      <th>Var5</th>\n",
       "      <th>Var1</th>\n",
       "      <th>...</th>\n",
       "      <th>Source</th>\n",
       "      <th>Var4</th>\n",
       "      <th>LoggedIn</th>\n",
       "      <th>Disbursed</th>\n",
       "      <th>DOB_year</th>\n",
       "      <th>DOB_month</th>\n",
       "      <th>DOB_day</th>\n",
       "      <th>LCD_year</th>\n",
       "      <th>LCD_month</th>\n",
       "      <th>LCD_day</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Female</td>\n",
       "      <td>Delhi</td>\n",
       "      <td>20000</td>\n",
       "      <td>300000.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>HDFC Bank</td>\n",
       "      <td>N</td>\n",
       "      <td>0</td>\n",
       "      <td>HBXX</td>\n",
       "      <td>...</td>\n",
       "      <td>S122</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1978</td>\n",
       "      <td>5</td>\n",
       "      <td>23</td>\n",
       "      <td>2015</td>\n",
       "      <td>5</td>\n",
       "      <td>15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Male</td>\n",
       "      <td>Mumbai</td>\n",
       "      <td>35000</td>\n",
       "      <td>200000.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>ICICI Bank</td>\n",
       "      <td>Y</td>\n",
       "      <td>13</td>\n",
       "      <td>HBXA</td>\n",
       "      <td>...</td>\n",
       "      <td>S122</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1985</td>\n",
       "      <td>10</td>\n",
       "      <td>7</td>\n",
       "      <td>2015</td>\n",
       "      <td>5</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Male</td>\n",
       "      <td>Panchkula</td>\n",
       "      <td>22500</td>\n",
       "      <td>600000.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>State Bank of India</td>\n",
       "      <td>Y</td>\n",
       "      <td>0</td>\n",
       "      <td>HBXX</td>\n",
       "      <td>...</td>\n",
       "      <td>S143</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1981</td>\n",
       "      <td>10</td>\n",
       "      <td>10</td>\n",
       "      <td>2015</td>\n",
       "      <td>5</td>\n",
       "      <td>19</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Male</td>\n",
       "      <td>Saharsa</td>\n",
       "      <td>35000</td>\n",
       "      <td>1000000.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>State Bank of India</td>\n",
       "      <td>Y</td>\n",
       "      <td>10</td>\n",
       "      <td>HBXX</td>\n",
       "      <td>...</td>\n",
       "      <td>S143</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1987</td>\n",
       "      <td>11</td>\n",
       "      <td>30</td>\n",
       "      <td>2015</td>\n",
       "      <td>5</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Male</td>\n",
       "      <td>Bengaluru</td>\n",
       "      <td>100000</td>\n",
       "      <td>500000.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>25000.0</td>\n",
       "      <td>HDFC Bank</td>\n",
       "      <td>Y</td>\n",
       "      <td>17</td>\n",
       "      <td>HBXX</td>\n",
       "      <td>...</td>\n",
       "      <td>S134</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1984</td>\n",
       "      <td>2</td>\n",
       "      <td>17</td>\n",
       "      <td>2015</td>\n",
       "      <td>5</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 28 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   Gender       City  Monthly_Income  Loan_Amount_Applied  \\\n",
       "0  Female      Delhi           20000             300000.0   \n",
       "1    Male     Mumbai           35000             200000.0   \n",
       "2    Male  Panchkula           22500             600000.0   \n",
       "3    Male    Saharsa           35000            1000000.0   \n",
       "4    Male  Bengaluru          100000             500000.0   \n",
       "\n",
       "   Loan_Tenure_Applied  Existing_EMI       Salary_Account Mobile_Verified  \\\n",
       "0                  5.0           0.0            HDFC Bank               N   \n",
       "1                  2.0           0.0           ICICI Bank               Y   \n",
       "2                  4.0           0.0  State Bank of India               Y   \n",
       "3                  5.0           0.0  State Bank of India               Y   \n",
       "4                  2.0       25000.0            HDFC Bank               Y   \n",
       "\n",
       "  Var5  Var1  ...  Source  Var4  LoggedIn  Disbursed  DOB_year DOB_month  \\\n",
       "0    0  HBXX  ...    S122     1         0          0      1978         5   \n",
       "1   13  HBXA  ...    S122     3         0          0      1985        10   \n",
       "2    0  HBXX  ...    S143     1         0          0      1981        10   \n",
       "3   10  HBXX  ...    S143     3         0          0      1987        11   \n",
       "4   17  HBXX  ...    S134     3         1          0      1984         2   \n",
       "\n",
       "  DOB_day LCD_year LCD_month  LCD_day  \n",
       "0      23     2015         5       15  \n",
       "1       7     2015         5        4  \n",
       "2      10     2015         5       19  \n",
       "3      30     2015         5        9  \n",
       "4      17     2015         5       20  \n",
       "\n",
       "[5 rows x 28 columns]"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_raw.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-04T08:42:32.913811Z",
     "iopub.status.busy": "2021-09-04T08:42:32.913376Z",
     "iopub.status.idle": "2021-09-04T08:42:32.980008Z",
     "shell.execute_reply": "2021-09-04T08:42:32.978773Z",
     "shell.execute_reply.started": "2021-09-04T08:42:32.913767Z"
    },
    "id": "4FDE9B7412E342BE8254870627165989",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "df_raw=pd.get_dummies(df_raw,columns=['Gender','Mobile_Verified',\n",
    "                                      'Filled_Form','Device_Type'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-04T08:42:32.981343Z",
     "iopub.status.busy": "2021-09-04T08:42:32.981012Z",
     "iopub.status.idle": "2021-09-04T08:42:33.053176Z",
     "shell.execute_reply": "2021-09-04T08:42:33.052298Z",
     "shell.execute_reply.started": "2021-09-04T08:42:32.981313Z"
    },
    "id": "E7F1449301F24B21B2E7ADA0815F9BDC",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>City</th>\n",
       "      <th>Monthly_Income</th>\n",
       "      <th>Loan_Amount_Applied</th>\n",
       "      <th>Loan_Tenure_Applied</th>\n",
       "      <th>Existing_EMI</th>\n",
       "      <th>Salary_Account</th>\n",
       "      <th>Var5</th>\n",
       "      <th>Var1</th>\n",
       "      <th>Loan_Amount_Submitted</th>\n",
       "      <th>Loan_Tenure_Submitted</th>\n",
       "      <th>...</th>\n",
       "      <th>Gender_Male</th>\n",
       "      <th>Mobile_Verified_0</th>\n",
       "      <th>Mobile_Verified_N</th>\n",
       "      <th>Mobile_Verified_Y</th>\n",
       "      <th>Filled_Form_Mobile</th>\n",
       "      <th>Filled_Form_N</th>\n",
       "      <th>Filled_Form_Y</th>\n",
       "      <th>Device_Type_G</th>\n",
       "      <th>Device_Type_Mobile</th>\n",
       "      <th>Device_Type_Web-browser</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Delhi</td>\n",
       "      <td>20000</td>\n",
       "      <td>300000.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>HDFC Bank</td>\n",
       "      <td>0</td>\n",
       "      <td>HBXX</td>\n",
       "      <td>395010.590188</td>\n",
       "      <td>3.891369</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Mumbai</td>\n",
       "      <td>35000</td>\n",
       "      <td>200000.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>ICICI Bank</td>\n",
       "      <td>13</td>\n",
       "      <td>HBXA</td>\n",
       "      <td>200000.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Panchkula</td>\n",
       "      <td>22500</td>\n",
       "      <td>600000.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>State Bank of India</td>\n",
       "      <td>0</td>\n",
       "      <td>HBXX</td>\n",
       "      <td>450000.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Saharsa</td>\n",
       "      <td>35000</td>\n",
       "      <td>1000000.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>State Bank of India</td>\n",
       "      <td>10</td>\n",
       "      <td>HBXX</td>\n",
       "      <td>920000.000000</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Bengaluru</td>\n",
       "      <td>100000</td>\n",
       "      <td>500000.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>25000.0</td>\n",
       "      <td>HDFC Bank</td>\n",
       "      <td>17</td>\n",
       "      <td>HBXX</td>\n",
       "      <td>500000.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 35 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        City  Monthly_Income  Loan_Amount_Applied  Loan_Tenure_Applied  \\\n",
       "0      Delhi           20000             300000.0                  5.0   \n",
       "1     Mumbai           35000             200000.0                  2.0   \n",
       "2  Panchkula           22500             600000.0                  4.0   \n",
       "3    Saharsa           35000            1000000.0                  5.0   \n",
       "4  Bengaluru          100000             500000.0                  2.0   \n",
       "\n",
       "   Existing_EMI       Salary_Account Var5  Var1  Loan_Amount_Submitted  \\\n",
       "0           0.0            HDFC Bank    0  HBXX          395010.590188   \n",
       "1           0.0           ICICI Bank   13  HBXA          200000.000000   \n",
       "2           0.0  State Bank of India    0  HBXX          450000.000000   \n",
       "3           0.0  State Bank of India   10  HBXX          920000.000000   \n",
       "4       25000.0            HDFC Bank   17  HBXX          500000.000000   \n",
       "\n",
       "   Loan_Tenure_Submitted  ...  Gender_Male  Mobile_Verified_0  \\\n",
       "0               3.891369  ...            0                  0   \n",
       "1               2.000000  ...            1                  0   \n",
       "2               4.000000  ...            1                  0   \n",
       "3               5.000000  ...            1                  0   \n",
       "4               2.000000  ...            1                  0   \n",
       "\n",
       "   Mobile_Verified_N Mobile_Verified_Y Filled_Form_Mobile  Filled_Form_N  \\\n",
       "0                  1                 0                  0              1   \n",
       "1                  0                 1                  0              1   \n",
       "2                  0                 1                  0              1   \n",
       "3                  0                 1                  0              1   \n",
       "4                  0                 1                  0              1   \n",
       "\n",
       "   Filled_Form_Y  Device_Type_G  Device_Type_Mobile  Device_Type_Web-browser  \n",
       "0              0              0                   0                        1  \n",
       "1              0              0                   0                        1  \n",
       "2              0              0                   0                        1  \n",
       "3              0              0                   0                        1  \n",
       "4              0              0                   0                        1  \n",
       "\n",
       "[5 rows x 35 columns]"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_raw.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-04T08:42:33.054826Z",
     "iopub.status.busy": "2021-09-04T08:42:33.054406Z",
     "iopub.status.idle": "2021-09-04T08:42:34.328811Z",
     "shell.execute_reply": "2021-09-04T08:42:34.327410Z",
     "shell.execute_reply.started": "2021-09-04T08:42:33.054795Z"
    },
    "id": "DA12082F1DED4FD8826EF00ED99105B5",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from category_encoders import TargetEncoder"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-04T08:42:34.330675Z",
     "iopub.status.busy": "2021-09-04T08:42:34.330346Z",
     "iopub.status.idle": "2021-09-04T08:42:35.227804Z",
     "shell.execute_reply": "2021-09-04T08:42:35.226895Z",
     "shell.execute_reply.started": "2021-09-04T08:42:34.330644Z"
    },
    "id": "E0DCF6BFAD5A477485B88883CADC1100",
    "jupyter": {},
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\Administrator\\anaconda3\\lib\\site-packages\\category_encoders\\utils.py:21: FutureWarning: is_categorical is deprecated and will be removed in a future version.  Use is_categorical_dtype instead\n",
      "  elif pd.api.types.is_categorical(cols):\n"
     ]
    }
   ],
   "source": [
    "#进行Target编码\n",
    "target_lst=['City','Salary_Account','Var5','Var1',\n",
    "            'Var2','Source','DOB_year','DOB_month',\n",
    "            'DOB_day','LCD_year','LCD_month','LCD_day']\n",
    "encoder_target=TargetEncoder(cols=target_lst)\n",
    "encoder_target.fit(df_raw, df_raw['Disbursed'])\n",
    "df_raw = encoder_target.transform(df_raw)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-04T08:42:35.229582Z",
     "iopub.status.busy": "2021-09-04T08:42:35.229114Z",
     "iopub.status.idle": "2021-09-04T08:42:35.287884Z",
     "shell.execute_reply": "2021-09-04T08:42:35.287112Z",
     "shell.execute_reply.started": "2021-09-04T08:42:35.229534Z"
    },
    "id": "751CDC170015434ABB643943F80FB1B1",
    "jupyter": {},
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>City</th>\n",
       "      <th>Monthly_Income</th>\n",
       "      <th>Loan_Amount_Applied</th>\n",
       "      <th>Loan_Tenure_Applied</th>\n",
       "      <th>Existing_EMI</th>\n",
       "      <th>Salary_Account</th>\n",
       "      <th>Var5</th>\n",
       "      <th>Var1</th>\n",
       "      <th>Loan_Amount_Submitted</th>\n",
       "      <th>Loan_Tenure_Submitted</th>\n",
       "      <th>...</th>\n",
       "      <th>Gender_Male</th>\n",
       "      <th>Mobile_Verified_0</th>\n",
       "      <th>Mobile_Verified_N</th>\n",
       "      <th>Mobile_Verified_Y</th>\n",
       "      <th>Filled_Form_Mobile</th>\n",
       "      <th>Filled_Form_N</th>\n",
       "      <th>Filled_Form_Y</th>\n",
       "      <th>Device_Type_G</th>\n",
       "      <th>Device_Type_Mobile</th>\n",
       "      <th>Device_Type_Web-browser</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2.147362e-02</td>\n",
       "      <td>20000</td>\n",
       "      <td>300000.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.021079</td>\n",
       "      <td>0.005192</td>\n",
       "      <td>0.011502</td>\n",
       "      <td>3.950106e+05</td>\n",
       "      <td>3.891369</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1.565540e-02</td>\n",
       "      <td>35000</td>\n",
       "      <td>200000.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.022514</td>\n",
       "      <td>0.041953</td>\n",
       "      <td>0.045690</td>\n",
       "      <td>2.000000e+05</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1.851503e-16</td>\n",
       "      <td>22500</td>\n",
       "      <td>600000.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.009457</td>\n",
       "      <td>0.005192</td>\n",
       "      <td>0.011502</td>\n",
       "      <td>4.500000e+05</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>6.937846e-04</td>\n",
       "      <td>35000</td>\n",
       "      <td>1000000.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.009457</td>\n",
       "      <td>0.023074</td>\n",
       "      <td>0.011502</td>\n",
       "      <td>9.200000e+05</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1.653732e-02</td>\n",
       "      <td>100000</td>\n",
       "      <td>500000.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>25000.0</td>\n",
       "      <td>0.021079</td>\n",
       "      <td>0.051449</td>\n",
       "      <td>0.011502</td>\n",
       "      <td>5.000000e+05</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>87015</th>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>71901</td>\n",
       "      <td>1000000.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>14500.0</td>\n",
       "      <td>0.022514</td>\n",
       "      <td>0.022797</td>\n",
       "      <td>0.011502</td>\n",
       "      <td>3.950106e+05</td>\n",
       "      <td>3.891369</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>87016</th>\n",
       "      <td>6.097561e-03</td>\n",
       "      <td>16000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.003953</td>\n",
       "      <td>0.001553</td>\n",
       "      <td>0.003349</td>\n",
       "      <td>2.400000e+05</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>87017</th>\n",
       "      <td>1.653732e-02</td>\n",
       "      <td>118000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.007229</td>\n",
       "      <td>0.015905</td>\n",
       "      <td>0.011502</td>\n",
       "      <td>1.200000e+06</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>87018</th>\n",
       "      <td>1.653732e-02</td>\n",
       "      <td>98930</td>\n",
       "      <td>800000.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>13660.0</td>\n",
       "      <td>0.022514</td>\n",
       "      <td>0.046392</td>\n",
       "      <td>0.011502</td>\n",
       "      <td>8.000000e+05</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>87019</th>\n",
       "      <td>1.565540e-02</td>\n",
       "      <td>42300</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.002040</td>\n",
       "      <td>0.030445</td>\n",
       "      <td>0.045690</td>\n",
       "      <td>6.900000e+05</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>87020 rows × 35 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "               City  Monthly_Income  Loan_Amount_Applied  Loan_Tenure_Applied  \\\n",
       "0      2.147362e-02           20000             300000.0                  5.0   \n",
       "1      1.565540e-02           35000             200000.0                  2.0   \n",
       "2      1.851503e-16           22500             600000.0                  4.0   \n",
       "3      6.937846e-04           35000            1000000.0                  5.0   \n",
       "4      1.653732e-02          100000             500000.0                  2.0   \n",
       "...             ...             ...                  ...                  ...   \n",
       "87015  0.000000e+00           71901            1000000.0                  5.0   \n",
       "87016  6.097561e-03           16000                  0.0                  0.0   \n",
       "87017  1.653732e-02          118000                  0.0                  0.0   \n",
       "87018  1.653732e-02           98930             800000.0                  5.0   \n",
       "87019  1.565540e-02           42300                  0.0                  0.0   \n",
       "\n",
       "       Existing_EMI  Salary_Account      Var5      Var1  \\\n",
       "0               0.0        0.021079  0.005192  0.011502   \n",
       "1               0.0        0.022514  0.041953  0.045690   \n",
       "2               0.0        0.009457  0.005192  0.011502   \n",
       "3               0.0        0.009457  0.023074  0.011502   \n",
       "4           25000.0        0.021079  0.051449  0.011502   \n",
       "...             ...             ...       ...       ...   \n",
       "87015       14500.0        0.022514  0.022797  0.011502   \n",
       "87016           0.0        0.003953  0.001553  0.003349   \n",
       "87017           0.0        0.007229  0.015905  0.011502   \n",
       "87018       13660.0        0.022514  0.046392  0.011502   \n",
       "87019           0.0        0.002040  0.030445  0.045690   \n",
       "\n",
       "       Loan_Amount_Submitted  Loan_Tenure_Submitted  ...  Gender_Male  \\\n",
       "0               3.950106e+05               3.891369  ...            0   \n",
       "1               2.000000e+05               2.000000  ...            1   \n",
       "2               4.500000e+05               4.000000  ...            1   \n",
       "3               9.200000e+05               5.000000  ...            1   \n",
       "4               5.000000e+05               2.000000  ...            1   \n",
       "...                      ...                    ...  ...          ...   \n",
       "87015           3.950106e+05               3.891369  ...            0   \n",
       "87016           2.400000e+05               4.000000  ...            0   \n",
       "87017           1.200000e+06               4.000000  ...            1   \n",
       "87018           8.000000e+05               5.000000  ...            1   \n",
       "87019           6.900000e+05               4.000000  ...            1   \n",
       "\n",
       "       Mobile_Verified_0  Mobile_Verified_N  Mobile_Verified_Y  \\\n",
       "0                      0                  1                  0   \n",
       "1                      0                  0                  1   \n",
       "2                      0                  0                  1   \n",
       "3                      0                  0                  1   \n",
       "4                      0                  0                  1   \n",
       "...                  ...                ...                ...   \n",
       "87015                  0                  1                  0   \n",
       "87016                  0                  0                  1   \n",
       "87017                  0                  0                  1   \n",
       "87018                  0                  0                  1   \n",
       "87019                  0                  0                  1   \n",
       "\n",
       "       Filled_Form_Mobile  Filled_Form_N  Filled_Form_Y  Device_Type_G  \\\n",
       "0                       0              1              0              0   \n",
       "1                       0              1              0              0   \n",
       "2                       0              1              0              0   \n",
       "3                       0              1              0              0   \n",
       "4                       0              1              0              0   \n",
       "...                   ...            ...            ...            ...   \n",
       "87015                   0              1              0              0   \n",
       "87016                   0              0              1              0   \n",
       "87017                   0              1              0              0   \n",
       "87018                   0              1              0              0   \n",
       "87019                   0              1              0              0   \n",
       "\n",
       "       Device_Type_Mobile  Device_Type_Web-browser  \n",
       "0                       0                        1  \n",
       "1                       0                        1  \n",
       "2                       0                        1  \n",
       "3                       0                        1  \n",
       "4                       0                        1  \n",
       "...                   ...                      ...  \n",
       "87015                   0                        1  \n",
       "87016                   1                        0  \n",
       "87017                   1                        0  \n",
       "87018                   0                        1  \n",
       "87019                   0                        1  \n",
       "\n",
       "[87020 rows x 35 columns]"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_raw"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "7C0B0CE4F5954CFF9F5FB20CD6E9DFDD",
    "jupyter": {},
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "上面已经完成了数据集的处理，保存一下。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-04T08:42:35.289556Z",
     "iopub.status.busy": "2021-09-04T08:42:35.289103Z",
     "iopub.status.idle": "2021-09-04T08:42:38.714626Z",
     "shell.execute_reply": "2021-09-04T08:42:38.713432Z",
     "shell.execute_reply.started": "2021-09-04T08:42:35.289512Z"
    },
    "id": "17DD2AE878B94B2381FEA95B17B9454D",
    "jupyter": {},
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "df_raw.to_csv('features.csv',index=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "CD1BE37EE1BF4BA291BBCD9AB4957C13",
    "jupyter": {},
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "下面进行训练集、测试集分割。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-04T08:42:38.716444Z",
     "iopub.status.busy": "2021-09-04T08:42:38.716130Z",
     "iopub.status.idle": "2021-09-04T08:42:38.869674Z",
     "shell.execute_reply": "2021-09-04T08:42:38.868382Z",
     "shell.execute_reply.started": "2021-09-04T08:42:38.716415Z"
    },
    "id": "A97872494F38464A8C1B6F36FDC7F87E",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "from sklearn.model_selection import train_test_split\n",
    "seed = 12345\n",
    "\n",
    "df_raw=pd.read_csv('features.csv')\n",
    "#选择所有自变量做为训练特征\n",
    "X = df_raw.drop(columns=['Disbursed'])\n",
    "Y = df_raw['Disbursed']\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, Y, test_size=0.20, random_state=seed)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {
    "id": "64CCE9DAFD68496B84A92671FF9E6D92",
    "jupyter": {},
    "notebookId": "613339d986080500170052d9",
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>City</th>\n",
       "      <th>Monthly_Income</th>\n",
       "      <th>Loan_Amount_Applied</th>\n",
       "      <th>Loan_Tenure_Applied</th>\n",
       "      <th>Existing_EMI</th>\n",
       "      <th>Salary_Account</th>\n",
       "      <th>Var5</th>\n",
       "      <th>Var1</th>\n",
       "      <th>Loan_Amount_Submitted</th>\n",
       "      <th>Loan_Tenure_Submitted</th>\n",
       "      <th>...</th>\n",
       "      <th>Gender_Male</th>\n",
       "      <th>Mobile_Verified_0</th>\n",
       "      <th>Mobile_Verified_N</th>\n",
       "      <th>Mobile_Verified_Y</th>\n",
       "      <th>Filled_Form_Mobile</th>\n",
       "      <th>Filled_Form_N</th>\n",
       "      <th>Filled_Form_Y</th>\n",
       "      <th>Device_Type_G</th>\n",
       "      <th>Device_Type_Mobile</th>\n",
       "      <th>Device_Type_Web-browser</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2.147362e-02</td>\n",
       "      <td>20000</td>\n",
       "      <td>300000.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.021079</td>\n",
       "      <td>0.005192</td>\n",
       "      <td>0.011502</td>\n",
       "      <td>395010.590188</td>\n",
       "      <td>3.891369</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1.565540e-02</td>\n",
       "      <td>35000</td>\n",
       "      <td>200000.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.022514</td>\n",
       "      <td>0.041953</td>\n",
       "      <td>0.045690</td>\n",
       "      <td>200000.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1.851503e-16</td>\n",
       "      <td>22500</td>\n",
       "      <td>600000.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.009457</td>\n",
       "      <td>0.005192</td>\n",
       "      <td>0.011502</td>\n",
       "      <td>450000.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>6.937846e-04</td>\n",
       "      <td>35000</td>\n",
       "      <td>1000000.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.009457</td>\n",
       "      <td>0.023074</td>\n",
       "      <td>0.011502</td>\n",
       "      <td>920000.000000</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1.653732e-02</td>\n",
       "      <td>100000</td>\n",
       "      <td>500000.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>25000.0</td>\n",
       "      <td>0.021079</td>\n",
       "      <td>0.051449</td>\n",
       "      <td>0.011502</td>\n",
       "      <td>500000.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 35 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "           City  Monthly_Income  Loan_Amount_Applied  Loan_Tenure_Applied  \\\n",
       "0  2.147362e-02           20000             300000.0                  5.0   \n",
       "1  1.565540e-02           35000             200000.0                  2.0   \n",
       "2  1.851503e-16           22500             600000.0                  4.0   \n",
       "3  6.937846e-04           35000            1000000.0                  5.0   \n",
       "4  1.653732e-02          100000             500000.0                  2.0   \n",
       "\n",
       "   Existing_EMI  Salary_Account      Var5      Var1  Loan_Amount_Submitted  \\\n",
       "0           0.0        0.021079  0.005192  0.011502          395010.590188   \n",
       "1           0.0        0.022514  0.041953  0.045690          200000.000000   \n",
       "2           0.0        0.009457  0.005192  0.011502          450000.000000   \n",
       "3           0.0        0.009457  0.023074  0.011502          920000.000000   \n",
       "4       25000.0        0.021079  0.051449  0.011502          500000.000000   \n",
       "\n",
       "   Loan_Tenure_Submitted  ...  Gender_Male  Mobile_Verified_0  \\\n",
       "0               3.891369  ...            0                  0   \n",
       "1               2.000000  ...            1                  0   \n",
       "2               4.000000  ...            1                  0   \n",
       "3               5.000000  ...            1                  0   \n",
       "4               2.000000  ...            1                  0   \n",
       "\n",
       "   Mobile_Verified_N  Mobile_Verified_Y  Filled_Form_Mobile  Filled_Form_N  \\\n",
       "0                  1                  0                   0              1   \n",
       "1                  0                  1                   0              1   \n",
       "2                  0                  1                   0              1   \n",
       "3                  0                  1                   0              1   \n",
       "4                  0                  1                   0              1   \n",
       "\n",
       "   Filled_Form_Y  Device_Type_G  Device_Type_Mobile  Device_Type_Web-browser  \n",
       "0              0              0                   0                        1  \n",
       "1              0              0                   0                        1  \n",
       "2              0              0                   0                        1  \n",
       "3              0              0                   0                        1  \n",
       "4              0              0                   0                        1  \n",
       "\n",
       "[5 rows x 35 columns]"
      ]
     },
     "execution_count": 73,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_raw.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "A4BADFC1B62B4B7F95029AF55D32050E",
    "jupyter": {},
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## 作业1\n",
    "对训练集数据构建XGBoost模型，并计算测试集的准确率和AUC值。可以尝试调节参数，感受不同参数对模型的影响。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.采用模型缺省值"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-04T08:42:38.871462Z",
     "iopub.status.busy": "2021-09-04T08:42:38.871137Z",
     "iopub.status.idle": "2021-09-04T08:42:41.418021Z",
     "shell.execute_reply": "2021-09-04T08:42:41.416620Z",
     "shell.execute_reply.started": "2021-09-04T08:42:38.871431Z"
    },
    "id": "6A2907B1FC884A16933F0960633967E5",
    "jupyter": {},
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\xgboost\\sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
      "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "XGBoost, time cost: 17.886097, Accuracy: 0.988853, Precision: 0.988853, AUC: 0.803595\n"
     ]
    }
   ],
   "source": [
    "import xgboost as xgb\n",
    "import time\n",
    "from sklearn.metrics import accuracy_score,precision_score\n",
    "from sklearn.metrics import roc_auc_score\n",
    "\n",
    "model = xgb.XGBClassifier()\n",
    "eval_set = [(X_test, y_test)]\n",
    "start = time.time()\n",
    "model.fit(X_train, y_train, \n",
    " early_stopping_rounds=10, \n",
    " eval_metric=\"logloss\", \n",
    " eval_set=eval_set, verbose=False)\n",
    "end=time.time()\n",
    "y_pred = model.predict(X_test)\n",
    "predictions = [value for value in y_pred]\n",
    "accuracy = accuracy_score(y_test, predictions)\n",
    "precision= precision_score(y_test, predictions,average='micro')\n",
    "auc=roc_auc_score(y_test, predictions)\n",
    "print(\"XGBoost, time cost: %f, Accuracy: %f, Precision: %f, AUC: %f\" % (end-start, accuracy,precision, auc))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.改变超参数1\n",
    "改变学习率、最大深度、subsample、colsample_bytree。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-04T08:42:41.420986Z",
     "iopub.status.busy": "2021-09-04T08:42:41.420316Z",
     "iopub.status.idle": "2021-09-04T08:42:45.824325Z",
     "shell.execute_reply": "2021-09-04T08:42:45.822968Z",
     "shell.execute_reply.started": "2021-09-04T08:42:41.420930Z"
    },
    "id": "96086BCECC21482FABE39F3AC0E007CB",
    "jupyter": {},
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\xgboost\\sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
      "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "XGBoost, time cost: 33.932403, Accuracy: 0.989428, Precision: 0.989428, AUC: 0.819783\n"
     ]
    }
   ],
   "source": [
    "model2 = xgb.XGBClassifier(\n",
    "     learning_rate =0.1,   #default:0.3\n",
    "     max_depth=5,          #default:6\n",
    "     min_child_weight=1,   #default:1\n",
    "     gamma=0,              #default:0\n",
    "     subsample=0.8,        #default:1\n",
    "     colsample_bytree=0.8, #default:1\n",
    "     objective= 'binary:logistic',\n",
    "     nthread=4,\n",
    "     scale_pos_weight=1,\n",
    "     seed=27)\n",
    "eval_set = [(X_test, y_test)]\n",
    "start = time.time()\n",
    "model2.fit(X_train, y_train, \n",
    " early_stopping_rounds=10, \n",
    " eval_metric=\"logloss\", \n",
    " eval_set=eval_set, verbose=False)\n",
    "end=time.time()\n",
    "y_pred = model2.predict(X_test)\n",
    "predictions = [value for value in y_pred]\n",
    "accuracy = accuracy_score(y_test, predictions)\n",
    "precision= precision_score(y_test, predictions,average='micro')\n",
    "auc=roc_auc_score(y_test, predictions)\n",
    "print(\"XGBoost, time cost: %f, Accuracy: %f, Precision: %f, AUC: %f\" % (end-start, accuracy,precision, auc))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.改变超参数2\n",
    "改变最大深度、subsample、colsample_bytree。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-04T08:42:45.825989Z",
     "iopub.status.busy": "2021-09-04T08:42:45.825684Z",
     "iopub.status.idle": "2021-09-04T08:42:49.135819Z",
     "shell.execute_reply": "2021-09-04T08:42:49.135031Z",
     "shell.execute_reply.started": "2021-09-04T08:42:45.825958Z"
    },
    "id": "255EB524BDF644BF897B0EED87F3CAA2",
    "jupyter": {},
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\xgboost\\sklearn.py:1146: UserWarning: The use of label encoder in XGBClassifier is deprecated and will be removed in a future release. To remove this warning, do the following: 1) Pass option use_label_encoder=False when constructing XGBClassifier object; and 2) Encode your labels (y) as integers starting with 0, i.e. 0, 1, 2, ..., [num_class - 1].\n",
      "  warnings.warn(label_encoder_deprecation_msg, UserWarning)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "XGBoost, time cost: 28.424328, Accuracy: 0.989658, Precision: 0.989658, AUC: 0.811951\n"
     ]
    }
   ],
   "source": [
    "model3 = xgb.XGBClassifier(\n",
    "     learning_rate =0.1,   #default:0.3\n",
    "     max_depth=4,          #default:6\n",
    "     min_child_weight=1,   #default:1\n",
    "     gamma=0,              #default:0\n",
    "     subsample=0.75,        #default:1\n",
    "     colsample_bytree=0.75, #default:1\n",
    "     objective= 'binary:logistic',\n",
    "     nthread=4,\n",
    "     scale_pos_weight=1,\n",
    "     seed=27)\n",
    "eval_set = [(X_test, y_test)]\n",
    "start = time.time()\n",
    "model3.fit(X_train, y_train, \n",
    " early_stopping_rounds=10, \n",
    " eval_metric=\"logloss\", \n",
    " eval_set=eval_set, verbose=False)\n",
    "end=time.time()\n",
    "y_pred = model3.predict(X_test)\n",
    "predictions = [value for value in y_pred]\n",
    "accuracy = accuracy_score(y_test, predictions)\n",
    "precision= precision_score(y_test, predictions,average='micro')\n",
    "auc=roc_auc_score(y_test, predictions)\n",
    "print(\"XGBoost, time cost: %f, Accuracy: %f, Precision: %f, AUC: %f\" % (end-start, accuracy,precision, auc))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "CD3F2B017A97487882FD538D46BD1FD1",
    "jupyter": {},
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## 作业2\n",
    "对训练数据使用SVM模型构建一个分类模型，理解SVM模型有哪些可以调整的超参数？并分别使用随机搜索，网格搜索，贝叶斯优化等方法对模型进行超参数调优，使得模型在测试集上准确率尽量高（最好达到95%或以上）。比较不同搜索方法的性能和结果。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.特征工程\n",
    "因为SVM模型很耗时，因此还是做一下特征工程。  \n",
    "1. 用LightGBM模型算一下特征重要性，获取重要性超过80%的特征用于SVM模型中，这样可以减轻计算量。  \n",
    "2. 另一方面整个模型有8万多条样本，对于SVM模型，计算量还是比较大，因此从中取5000条用于训练，800条用于测试。\n",
    "3. SVM模型中采用RBF径向基核，对于样本进行标准化处理。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_fea =  np.array(X_train)\n",
    "test_fea = np.array(X_test)\n",
    "#evaluate_fea = np.array(df_main[df_main.target==-1].drop(['Idx','target','ListingInfo1'],axis=1))\n",
    " \n",
    "# # reshape(-1,1)转成一列\n",
    "train_label = np.array(y_train).reshape(-1,1)\n",
    "test_label = np.array(y_test).reshape(-1,1)\n",
    "#evaluate_label = np.array(df_main[df_main.target==-1]['target']).reshape(-1,1)\n",
    " \n",
    " \n",
    "fea_names = list(X_train.columns)\n",
    "feature_importance_values = np.zeros(len(fea_names))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\validation.py:63: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  return f(*args, **kwargs)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training until validation scores don't improve for 100 rounds\n",
      "Early stopping, best iteration is:\n",
      "[78]\tvalid_0's auc: 0.987128\tvalid_0's binary_logloss: 0.0226557\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\validation.py:63: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  return f(*args, **kwargs)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training until validation scores don't improve for 100 rounds\n",
      "Early stopping, best iteration is:\n",
      "[78]\tvalid_0's auc: 0.987128\tvalid_0's binary_logloss: 0.0226557\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\validation.py:63: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  return f(*args, **kwargs)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training until validation scores don't improve for 100 rounds\n",
      "Early stopping, best iteration is:\n",
      "[78]\tvalid_0's auc: 0.987128\tvalid_0's binary_logloss: 0.0226557\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\validation.py:63: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  return f(*args, **kwargs)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training until validation scores don't improve for 100 rounds\n",
      "Early stopping, best iteration is:\n",
      "[78]\tvalid_0's auc: 0.987128\tvalid_0's binary_logloss: 0.0226557\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\validation.py:63: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  return f(*args, **kwargs)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training until validation scores don't improve for 100 rounds\n",
      "Early stopping, best iteration is:\n",
      "[78]\tvalid_0's auc: 0.987128\tvalid_0's binary_logloss: 0.0226557\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\validation.py:63: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  return f(*args, **kwargs)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training until validation scores don't improve for 100 rounds\n",
      "Early stopping, best iteration is:\n",
      "[78]\tvalid_0's auc: 0.987128\tvalid_0's binary_logloss: 0.0226557\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\validation.py:63: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  return f(*args, **kwargs)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training until validation scores don't improve for 100 rounds\n",
      "Early stopping, best iteration is:\n",
      "[78]\tvalid_0's auc: 0.987128\tvalid_0's binary_logloss: 0.0226557\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\validation.py:63: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  return f(*args, **kwargs)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training until validation scores don't improve for 100 rounds\n",
      "Early stopping, best iteration is:\n",
      "[78]\tvalid_0's auc: 0.987128\tvalid_0's binary_logloss: 0.0226557\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\validation.py:63: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  return f(*args, **kwargs)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training until validation scores don't improve for 100 rounds\n",
      "Early stopping, best iteration is:\n",
      "[78]\tvalid_0's auc: 0.987128\tvalid_0's binary_logloss: 0.0226557\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\validation.py:63: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n",
      "  return f(*args, **kwargs)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training until validation scores don't improve for 100 rounds\n",
      "Early stopping, best iteration is:\n",
      "[78]\tvalid_0's auc: 0.987128\tvalid_0's binary_logloss: 0.0226557\n"
     ]
    }
   ],
   "source": [
    "import lightgbm as lgb\n",
    "\n",
    "for i in np.arange(10):\n",
    "    model = lgb.LGBMClassifier(n_estimators=1000,\n",
    "                              learning_rate=0.05,\n",
    "                              n_jobs=-1,\n",
    "                              verbose=-1)\n",
    "    model.fit(train_fea,train_label,\n",
    "              eval_metric='auc',\n",
    "             eval_set = [(test_fea, test_label)],\n",
    "             early_stopping_rounds=100,\n",
    "              verbose = -1)\n",
    "    feature_importance_values += model.feature_importances_/10"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>feature</th>\n",
       "      <th>fea_importance</th>\n",
       "      <th>norm_importance</th>\n",
       "      <th>cum_importance</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Monthly_Income</td>\n",
       "      <td>225.0</td>\n",
       "      <td>0.096154</td>\n",
       "      <td>0.096154</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>LCD_day</td>\n",
       "      <td>191.0</td>\n",
       "      <td>0.081624</td>\n",
       "      <td>0.177778</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Existing_EMI</td>\n",
       "      <td>180.0</td>\n",
       "      <td>0.076923</td>\n",
       "      <td>0.254701</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>City</td>\n",
       "      <td>168.0</td>\n",
       "      <td>0.071795</td>\n",
       "      <td>0.326496</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>DOB_year</td>\n",
       "      <td>158.0</td>\n",
       "      <td>0.067521</td>\n",
       "      <td>0.394017</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>DOB_day</td>\n",
       "      <td>146.0</td>\n",
       "      <td>0.062393</td>\n",
       "      <td>0.456410</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Var5</td>\n",
       "      <td>137.0</td>\n",
       "      <td>0.058547</td>\n",
       "      <td>0.514957</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>DOB_month</td>\n",
       "      <td>135.0</td>\n",
       "      <td>0.057692</td>\n",
       "      <td>0.572650</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Salary_Account</td>\n",
       "      <td>134.0</td>\n",
       "      <td>0.057265</td>\n",
       "      <td>0.629915</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Source</td>\n",
       "      <td>127.0</td>\n",
       "      <td>0.054274</td>\n",
       "      <td>0.684188</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>EMI_Loan_Submitted</td>\n",
       "      <td>97.0</td>\n",
       "      <td>0.041453</td>\n",
       "      <td>0.725641</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Loan_Amount_Submitted</td>\n",
       "      <td>85.0</td>\n",
       "      <td>0.036325</td>\n",
       "      <td>0.761966</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>Processing_Fee</td>\n",
       "      <td>80.0</td>\n",
       "      <td>0.034188</td>\n",
       "      <td>0.796154</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>LoggedIn</td>\n",
       "      <td>78.0</td>\n",
       "      <td>0.033333</td>\n",
       "      <td>0.829487</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>Loan_Amount_Applied</td>\n",
       "      <td>65.0</td>\n",
       "      <td>0.027778</td>\n",
       "      <td>0.857265</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>Loan_Tenure_Submitted</td>\n",
       "      <td>59.0</td>\n",
       "      <td>0.025214</td>\n",
       "      <td>0.882479</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>Var4</td>\n",
       "      <td>53.0</td>\n",
       "      <td>0.022650</td>\n",
       "      <td>0.905128</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>Interest_Rate</td>\n",
       "      <td>52.0</td>\n",
       "      <td>0.022222</td>\n",
       "      <td>0.927350</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>LCD_month</td>\n",
       "      <td>44.0</td>\n",
       "      <td>0.018803</td>\n",
       "      <td>0.946154</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>Loan_Tenure_Applied</td>\n",
       "      <td>43.0</td>\n",
       "      <td>0.018376</td>\n",
       "      <td>0.964530</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>Var2</td>\n",
       "      <td>30.0</td>\n",
       "      <td>0.012821</td>\n",
       "      <td>0.977350</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>Var1</td>\n",
       "      <td>23.0</td>\n",
       "      <td>0.009829</td>\n",
       "      <td>0.987179</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>Mobile_Verified_Y</td>\n",
       "      <td>10.0</td>\n",
       "      <td>0.004274</td>\n",
       "      <td>0.991453</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>Filled_Form_N</td>\n",
       "      <td>9.0</td>\n",
       "      <td>0.003846</td>\n",
       "      <td>0.995299</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>Gender_Female</td>\n",
       "      <td>4.0</td>\n",
       "      <td>0.001709</td>\n",
       "      <td>0.997009</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>Mobile_Verified_N</td>\n",
       "      <td>4.0</td>\n",
       "      <td>0.001709</td>\n",
       "      <td>0.998718</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>Gender_Male</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0.000855</td>\n",
       "      <td>0.999573</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>Device_Type_Mobile</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.000427</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>LCD_year</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>Mobile_Verified_0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>Filled_Form_Mobile</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>Filled_Form_Y</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>Device_Type_G</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>33</th>\n",
       "      <td>Device_Type_Web-browser</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                    feature  fea_importance  norm_importance  cum_importance\n",
       "0            Monthly_Income           225.0         0.096154        0.096154\n",
       "1                   LCD_day           191.0         0.081624        0.177778\n",
       "2              Existing_EMI           180.0         0.076923        0.254701\n",
       "3                      City           168.0         0.071795        0.326496\n",
       "4                  DOB_year           158.0         0.067521        0.394017\n",
       "5                   DOB_day           146.0         0.062393        0.456410\n",
       "6                      Var5           137.0         0.058547        0.514957\n",
       "7                 DOB_month           135.0         0.057692        0.572650\n",
       "8            Salary_Account           134.0         0.057265        0.629915\n",
       "9                    Source           127.0         0.054274        0.684188\n",
       "10       EMI_Loan_Submitted            97.0         0.041453        0.725641\n",
       "11    Loan_Amount_Submitted            85.0         0.036325        0.761966\n",
       "12           Processing_Fee            80.0         0.034188        0.796154\n",
       "13                 LoggedIn            78.0         0.033333        0.829487\n",
       "14      Loan_Amount_Applied            65.0         0.027778        0.857265\n",
       "15    Loan_Tenure_Submitted            59.0         0.025214        0.882479\n",
       "16                     Var4            53.0         0.022650        0.905128\n",
       "17            Interest_Rate            52.0         0.022222        0.927350\n",
       "18                LCD_month            44.0         0.018803        0.946154\n",
       "19      Loan_Tenure_Applied            43.0         0.018376        0.964530\n",
       "20                     Var2            30.0         0.012821        0.977350\n",
       "21                     Var1            23.0         0.009829        0.987179\n",
       "22        Mobile_Verified_Y            10.0         0.004274        0.991453\n",
       "23            Filled_Form_N             9.0         0.003846        0.995299\n",
       "24            Gender_Female             4.0         0.001709        0.997009\n",
       "25        Mobile_Verified_N             4.0         0.001709        0.998718\n",
       "26              Gender_Male             2.0         0.000855        0.999573\n",
       "27       Device_Type_Mobile             1.0         0.000427        1.000000\n",
       "28                 LCD_year             0.0         0.000000        1.000000\n",
       "29        Mobile_Verified_0             0.0         0.000000        1.000000\n",
       "30       Filled_Form_Mobile             0.0         0.000000        1.000000\n",
       "31            Filled_Form_Y             0.0         0.000000        1.000000\n",
       "32            Device_Type_G             0.0         0.000000        1.000000\n",
       "33  Device_Type_Web-browser             0.0         0.000000        1.000000"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fea_imp_df2 = pd.DataFrame({'feature':fea_names,\n",
    "                           'fea_importance':feature_importance_values})\n",
    "fea_imp_df2 = fea_imp_df2.sort_values('fea_importance',ascending=False).reset_index(drop=True)\n",
    "fea_imp_df2['norm_importance'] = fea_imp_df2['fea_importance']/fea_imp_df2['fea_importance'].sum() # 特征重要性value的归一化\n",
    "fea_imp_df2['cum_importance'] = np.cumsum(fea_imp_df2['norm_importance'])# 特征重要性value的累加值\n",
    " \n",
    "fea_imp_df2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABAoAAAOnCAYAAACpgaSSAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAADQxUlEQVR4nOzdd5hdVdmG8ftNIxA6oYQaOkgVAlKCdJAuXaQXQRCUoqAUBUQEhU9FDRJRsCAdUYp0EkIAMZHeAyRUKaEGSEh5vz/2nslhmCQzk5nZU+7fdZ1r9l67vWfOmWTOM2utHZmJJEmSJEkSQI+qC5AkSZIkSR2HQYEkSZIkSapnUCBJkiRJkuoZFEiSJEmSpHoGBZIkSZIkqZ5BgSRJajUR0Tci+lRdx6xERM+qa5AkqaMyKJAkSS0SEZdGxGYNmk8HDmhk34UiYodWuOacEbFoRCwyG+dYAbiz9pzl19Mj4tjZrVGSpM6uV9UFSJKkziUiFgD2bMb+/YDrgKsj4gjgtHJTD2Bh4I1y/eLMPD0ibgYWAqYBCcxP8TvL28AnwEfAA8DZEXEw8J2ay60KPFUurwI8XbPt5My8uZESb4iIHzf1+UiS1NUZFEiSpOZaHdgCmAgQEYOAi8ttA4BJEXF0uX4w8Hvgusz8Tdl2UXlcf+DezFyl7sQRsSnwtwbX2wLoDdxa2xgRW2fmJRGxN9C/bE5gSiPLTzYICZaKiN2B24ANgdHA5k3+DkiS1IUZFEiSpOZaG1gKWBRYDLghM9cGiIhzgDGZeXG5vhPwc+DvETEc2C0zx8/k3B9Q9ByoNQHo00j7++XXVTJzYHm9MZk5KCJ6AY+Xy/2BW8rtOwPrAR8DPwGWpeil8ACwCDA1Ig4rz3tRZv66Sd8RSZK6EIMCSZLUXLsA+wBnApcC+0XEw+W2uh4FPwLeAx7KzAMi4mzgXmBCRNyfmRs2duLMfKj8oH4S8GrZvDjFMIW6ngeLUQxTOK8FtW9CMRxhPPBD4Crgq5l5b0ScDryXmb9swXklSeoyDAokSVJz7ZKZH0XEHMDxwOjMPKx2h4gYBeyRmWMjYheKD/5zUPzuMaBm1+UiYky5/J3MvKlcvg+4vVxuOPRgkwb1PFheD2DxiHiCopfAhLJ9GvDvcntf4D/A/sAo4IzMvLf53wJJkrougwJJktRcB0bEMcBA4FfA5JoeBXVWAm6OiMWA14FpmTmlHBJQ64XaOQpqvMH0iQi/QBEy1K2vULtjZu4FEBEDKeYcWDMzpzZWeGYeU971gMz8OCJ6RcTj5eaGQw+OyMyRjZ1HkqSuzKBAkiQ119vAXsAJwC2ZOSwi3imXx8L0HgXAEcDvgGeaeY3dgY3K5cUohh58qVxfBBhaXudmiqEJAGsCTwCjIwKKeQ1WBR4ptz+UmQfXXqScg+DX5blOx6EHkiQZFEiSpObJzKsAyg/jdcYCN0bE4Mx8r2bfHzSy76w8SxEw1A0nOJhiyMCF5foaFD0MyMzty/MfBRwKbJqZEyKiB3AJcHtm1t4+UZIkzYJBgSRJmm2ZeUtEfAx8WA4v6A9MbuHp1gUWAjYr19eh+J2lX80+7wN31KzfTdHTYHhEvFnu+yFwSAtrkCSp2zIokCRJrWU08BFFQPBvirkJ6kXEBsAVwGIRMZZiOMHC5TIUd0WYCKze4LyLlPvO2+B8ywJHUwxNGFAeO4EipLgb2Al4KCL+AfwjM0c1OP7LwJAG16mdo+DVzNy26U9fkqSuwaBAkiS1isz8iGKIwIy2P0AxAWKriYgtKeYYeILiTgnfzMynys0nRsSawN7A15g+lKGunnv4fCghSVK3F5lZdQ2SJEmSJKmD6FF1AZIkSZIkqeMwKJAkSZIkSfUMCiRJkiRJUj0nM+zE+vfvnwMHDqy6DEmSJElSGxg9evTbmblwe1/XoKATGzhwIKNGjZr1jpIkSZKkTicixlVxXYceSJIkSZKkevYo6MSeemU8637vz1WXIUmSJEmdxuifH1B1CR2ePQokSZIkSVI9gwJJkiRJklTPoECSJEmSJNUzKJAkSZIkSfW6TFAQEWMjYslG2i+NiP2qqEmSJEmSpM6mzYOCiMiIuLCR9j9ExJjZOO/qEXHI7FU3w3NPaYvzSpIkSZLU0bVHj4KpwLYRMXddQ0TMD2w6m+cdBGw0m+eQJEmSJEk12mvowS3APjXrBwO31+4QEcdExNMR8XxEDImI3mX72Ig4OSKeiohnImLViNgcOBfYMyL+W3Oa/ctzPBsRqzc4/1kRcXbN+mURsdvMio6IgRExJiJ+EREvRcQdETFnuW3liLg7Il6IiDvLtjUjYmT5HO6LiFXL9tPLIRD3RsT/ImLHiLgiIl6JiEtrrrdfRDweEc9FxDeb/N2VJEmSJKmVtFdQMAQ4AiAigiIouKRuY/nB/2BgfWAlYJG6/etk5qrA1cBxmXk3cBJwdWauU7Nbr8xcBbgCOLZBDZcDO5XX6w1sAtzchNqXBa4DBgK9gV0jomfZ9ovMXA74WkT0Aq4BTs7M5SmCjL/UnGdVYHPgGOBa4MLyuW5Vhh8rAQdR9JRYCzguIhZpQn2SJEmSJLWadgkKMvNx4OOIGARsCTwOvFmzy/bApZn5QWZOBX4PbFGz/cry63BgmZlc6vLy6whgqQY1PAFkRCxXnvuuzJzYhPJfy8wRmTkNuLe8/orA5Mz8Z3nut4CVgQmZObxs+wewRETMW57n1sycDNwHvJOZwzPz4/J7sSSwDUVA8DDwX2AeinDiMyLi8IgYFRGjpnz8YRPKlyRJkiSp6Xq147UuBA4HFgAuaKSOaTXrQTG3QZ1J5dfJQM+ZXGPiLPa7HNiZ4q/7lzeyvTGTapbrztsXaDjhYcPnUKeu7dPy69QG55xSnrMX8OfMPGFmxWTmUGAoQL/Fls0m1C9JkiRJUpO15+0Rr6WYwHDZzBzRYNvtwMERMW/Zrf8QinkNZuYToH85lKGprqAYfrAhcGczjmvoKWCRiNgUICIWL9v6R8QmZdtOwJjMnNDEc94L7F433KDu3JIkSZIktad261GQmZ9GxDXA641suzki1gIeovgL+/XUzGEwA3cDZwKjgXVmsW/ddV4sJyMckZktvgViZk6KiL2ACyNiPuC/mblH2TakbBsLHNCMc46KiCHAqIj4FLiRYqiFJEmSJEntJjK7V+/1iLgZODsz7626ltnVb7Flc5X9z6i6DEmSJEnqNEb/vMl/z61cRIzOzEHtfd32HHpQuYhYg2KSw5FV1yJJkiRJUkfUbYKCiDgWuAE4IstuFBHx7YgYU/N4utIiJUmSJEmqWHve9aBSmflL4JcN2i7g83dgkCRJkiSp2+o2PQokSZIkSdKsGRRIkiRJkqR63WboQVe06pILMaoTzdgpSZIkSer47FEgSZIkSZLqGRRIkiRJkqR6BgWSJEmSJKmeQYEkSZIkSarnZIad2KevP8FLZ65RdRmSJElSp7L0Dx+rugSpQ7NHgSRJkiRJqmdQIEmSJEmS6hkUSJIkSZKkegYFkiRJkiSpnkGBJEmSJEmqZ1DQjiJiWEQMrroOSZIkSZJmpNsHBRExpZG2ARHx14h4JSLGRcR9ZfvpETE+Ip4v238bEfO1f9WSJEmSJLWNbh8UNFR+8B8GjACWycxlgP1qdvlFZi4PrAC8CNwdEXO0e6GSJEmSJLUBg4LPOwK4OzMvysypAJn5QsOdMnNyZp4HPAN8vbETRUS/iLis7IHwT2Dumm0/i4gxZc+EfSJinoh4LyLmKbevGBGPtsUTlCRJkiRpRgwKPm8wcFsz9r8HWG8G234AfFj2QDgJWLVm242ZuQKwA/DTzPwQuB3Ytty+K3B5wxNGxOERMSoiRr3z0dRmlClJkiRJ0qwZFHxeApOasX9vYPIMtm0LDAHIzKeAh2q2fRIR5wA/AZYo2y4Hdi6Xvwpc8bniModm5qDMHLRgv57NKFOSJEmSpFkzKPi8x4FNmrH/VsDIGWzry2dDhDkAImI14ErgZuAbQJTbbwYGR8RSQGbmi82oQ5IkSZKk2WZQ8Hm/Bg6KiK/WNUTEGg13iog5IuI0YCHguhmc69/AIeX+GwBrle2rAs9STJi4Wd3OmTmRYijDuRRBgiRJkiRJ7cqgAHqWkwi+FxFvZ+b/gJ2A4yLifxHxPHBGzf7HlW3PAgsC22bm526xWDoNWC8ixgFHA/eV7bcB8wIvAV9ocMwVwJ7AVa3x5CRJkiRJao5eVRdQtcyMRtr+A2zaSPvpwOnNOPfr1PQYaGCjmuXac74DDC8DC0mSJEmS2pU9CjqQiAjgeODiqmuRJEmSJHVPBgWtJCKGRcSYmsfZLTjNa8CnOD+BJEmSJKki3X7oQWvJzM1a4RwDWqEUSZIkSZJazB4FkiRJkiSpnkGBJEmSJEmq59CDTqzPgNVY+oejqi5DkiRJktSF2KNAkiRJkiTVMyiQJEmSJEn1DAokSZIkSVI9gwJJkiRJklTPyQw7safffJqNf71x1WVIkiRJAIw8ZmTVJUhqBfYokCRJkiRJ9QwKJEmSJElSPYMCSZIkSZJUz6BAkiRJkiTVMyiQJEmSJEn1unRQEBGHR8Terb2vJEmSJEldVacMCiIiI2JM+Rgxo/0yc2hmXjmT8/yiqfu2oMaDIuL9mjqPK9vHRsS/Gtn/tIiYUrM+peE+kiRJkiS1tV5VF9BCUzNzhVY4zzHAca1wnhm5OjMPa6R97YgYmJljASKiJ7AX8HEb1iJJkiRJ0ix1yh4FDUXEAhHxakQsERG9IuKJiFglIk6PiFPLfU6NiHHlY82IeBDoWf61f6sG+w6LiDMj4pGIeCkiNinb546IKyLihYj4e0T8JyIGt6Dkq4Fv1KzvDIwE+s7ed0KSJEmSpNnTWYOCnhHxdPn4TWa+C5wNnAQcCtyUmU/X7RwRCwDfBVYGVgReyMz1KXsmZOYdjVxj0cxcCzgTOK1sOxn4IDOXA04FVptFnXvU1LllTfsfgK9HRO9y/UjgdzTh9SjnUhgVEaMmT5g8q90lSZIkSWqWzjz0YJUGbb8D/g1sDGzaYNsHwDjgl8BPM3NcE65xVfl1OPC9cnkryp4AmflERDw0i3NcM4OhB+OBEcAuEfEI0C8zH46IWRaVmUOBoQBzLz13zvIASZIkSZKaobP2KJiRTym670+rbczMqcD6wKPAyIjYuAnnmlR+nQz0LJfnKq9RpzctdyFwePn43WycR5IkSZKkVtOVgoJvAw9Q9AQ4tXZDRPQDFsrMIcB1wHrlpskRMX805U/5hQeBw8pzrg+s3dJiM/N+YEFgR4o5CyRJkiRJqlxnHXrQMyLG1KxvTnH3grUp/uL/ZERcUrN9LuDuiOgBvACcUbZfDDwF7NvE654KXB4R44B7gNHA1Jnsv2dEbFYu356ZRzbYfhGwcmZObOL1JUmSJElqU5HpMPeWiojngc0z86Uqrj/30nPnWt9bq4pLS5IkSZ8z8piRVZcgdSkRMTozB7X3dbvS0IM2FxFfjIj5y+V9gY+AlystSpIkSZKkVtRZhx5UZQXgnxGRFEMYvpaZGRHDgCVr9rsqM0+uokBJkiRJkmaHQUEzZObVNDLxYGZu1v7VSJIkSZLU+hx6IEmSJEmS6tmjoBNbZZFVnDBGkiRJktSq7FEgSZIkSZLqGRRIkiRJkqR6BgWSJEmSJKmeQYEkSZIkSarnZIad2IfPPMPwL29adRmSJElqA5veM7zqEiR1U/YokCRJkiRJ9QwKJEmSJElSPYMCSZIkSZJUz6BAkiRJkiTVMyiQJEmSJEn1DAraWUTsEBHHVV2HJEmSJEmNMShoQxHRJyLOiogXI2JsRDwPPJWZvyi3rx4Rh1RcpiRJkiRJ9QwK2tbfgEWBNTJzIPAl4K2a7YOAjSqoS5IkSZKkRvWquoCuKiLWB1YF1szMqQCZ+XZEHBQRg4HLgHOBvhGxDnAz0CMzTy6Pvwy4NjOvq+YZSJIkSZK6I3sUtJ3BwJ11IUFDmXk3cBJwdWauA1wO7AQQEb2BTSjCA0mSJEmS2o09CtpOApOavHPmExGREbEcsCJwV2ZObLhfRBwOHA6w6BxztFatkiRJkiQB9ihoS49T9ApojsuBnYHdyuXPycyhmTkoMwfN17v3bJYoSZIkSdJnGRS0nTsAIuLMcigBEbEkMH/NPp8A/SMiyvUrKIYfbAjc2X6lSpIkSZJUMChoI5mZwC4UwwheiYgxwLXA3DW73U0x4eHo8pgXgTmBEZk5pX0rliRJkiTJOQraVGa+Aewzk+1vAis3aH6PGQw7kCRJkiSprdmjoAOJiDWApYCRVdciSZIkSeqeDAo6iIg4FrgBOKIctiBJkiRJUrszKOggMvOXmTkwM++ruhZJkiRJUvdlUCBJkiRJkuoZFEiSJEmSpHre9aATm2flldn0nuFVlyFJkiRJ6kLsUSBJkiRJkuoZFEiSJEmSpHoGBZIkSZIkqZ5BgSRJkiRJqudkhp3Ym6+8z29OuKHqMiRJkjq1o8/fqeoSJKlDsUeBJEmSJEmqZ1AgSZIkSZLqGRRIkiRJkqR6BgWSJEmSJKmeQYEkSZIkSapnUCBJkiRJkup1m6AgIjIixkTEyxFxf0Ts0GD7pmX7K+U+50REr3LbZhHxcUQ8HxEvRcQp1TwLSZIkSZLaVpODgohYJCJ+GRFXleurRsSabVdaq5uamStk5lLAocCZEbE3QEQsD/wNODEzlwRWAZYBflxz/H2ZuTywBnB8RCzQ1gVHRLT1NSRJkiRJqtWcHgWXAU8Ba5frbwOXtHZB7SEznwS+DvysbPo+cE5mjii3fwQcARwWEXM1OHwx4FXgg4bnjYitIuK+mvVvRMQFEdEjIs6LiKci4pGI2Kjc/rWIeDoixkXEL8q2gWXPh8uAO1r3mUuSJEmSNHPNCQqWzMyLgKkAmfkWsGCbVNUOMvMZoG9ELAKsBQxvsP0D4AVgxbJpw4h4DngUGJKZUxs57d3AMuU5AXYFLgcOoujRsCqwN3Bhuf3F8torA3tGxMCyfSBwEbBVwwtExOERMSoiRk34+P3mPm1JkiRJkmaqOUHBMxGxLZAR0S8iTgRebqO62ksvYDLQB5jWyPYEPi2X78/MFYEBwCERsfnndi7Cg2uBHSNiXmB54AFge+BrEfE0cD1QFyS8CpxA0TNjbmCJsv3tzLwnM7ORawzNzEGZOWjuueZrwVOWJEmSJGnGmhMUHELRXb8/RUAwGDiwLYpqDxGxDvBGZr5L0UtgcIPt81LMU/BCbXtmvgPcAgyawakvB3YGdgCuKz/s9wKOzsxVyseAcqLEkeX5TwCeAOrmJJjQCk9RkiRJkqRma05QcHxmHpiZi2Tmgpm5c2a+2GaVtaGIWAu4FDixbDoXOCUi1iu3zw0MBS7MzEkNjp0T2BJ4qLFzZ+b9wAoUQwwuL5vvBb4REb0iondEbAjMD8wD/BPoC3SmiSElSZIkSV1Uc4KCLSJi/rYqpB30LCcJfAX4DUXwcSNAZj4BHAZcVG5/HHgG+EnN8RtFxBiKv/zfkpkzm2jwRmCFzHy0XP8t8D7FnASPA8tn5tvA1WXbeWW7JEmSJEmV6tWMff8HPBERw6npGp+Zh7d6VW0gM2d6q8HMvBW4dQbbhgEN734wM+8AV9Qc/wmwfyPnPYLi7goNrdCMa0mSJEmS1GqaExT8o3xoJiKiH3AojdyxQJIkSZKkjq7JQUFm/qktC+mMyqEItf4CHAWckZmd/Y4QkiRJkqRuqMlBQUQ8R3G7wM/IzJVataJOJDMbGyJwRrsXIkmSJElSK2nO0IParvS9KWb+X6Z1y5EkSZIkSVVqztCDcQ2axkTE31u5HjXDIkvOx9Hn71R1GZIkSZKkLqQ5Qw82qlntCawNDGzleiRJkiRJUoWaM/TgJzXLU4CxwNdatRpJkiRJklSp5gQFR2bm07UNEdFtJzKUJEmSJKkr6tGMff9ZuxIRPYA7WrccSZIkSZJUpVn2KIiI04F9gaUi4tmaTfMB97VRXZIkSZIkqQKRmTPfIWJuoD9wN7BZzaZPMvPNtitNs7LEQgvkUdttWXUZkiRJ9U756zVVlyBJXUZEjM7MQe193Vn2KMjMCcCEiPhCZn7SDjVJkiRJkqSKNGeOgp0j4sWImBQRn9Y92qwySZIkSZLU7poTFJwD7Aj8C1gC2A84qy2KkiRJkiRJ1WhOUDAlM58AngJWysyrgEPbpixJkiRJklSF5gQF10XEYOBa4M8RcTPwStuU1TVFxNiIWLLqOiRJkiRJmpEmBwWZeVJm3puZo4CdgD8AX2mzytpQRGREjImIlyPi/ojYocH2Tcv2V8p9zomIXuW2zSLi44h4PiJeiohTqnkWkiRJkiS1viYHBRGxSET8MiKuzMwngSeBZduutDY1NTNXyMylKIZPnBkRewNExPLA34ATM3NJYBVgGeDHNcffl5nLA2sAx0fEAu1bviRJkiRJbaM5Qw8uo5if4Ivl+tvAJa1eUTsrQ4+vAz8rm74PnJOZI8rtHwFHAIdFxFwNDl8MeBX4oLFzl+HKv8reC3+i5naUEfG3slfCmIjYIiJWKHswRLl964j4Z2s+V0mSJEmSZqU5QcGSmXkRMBUgM98CFmyTqtpZZj4D9I2IRYC1gOENtn8AvACsWDZtGBHPAY8CQzJz6gxO/X/A8MxcgSJUWbxm2+/KXgnHAj/MzDHA68C65fZdgctn97lJkiRJktQczQkKnomIbYGMiH4RcSLwchvVVYVewGSgDzCtke0JfFou35+ZKwIDgEMiYvMZnHMbYAhAZg4DXqu9XkT8H3A8xe0moQgGdi57FWwNfK5HQUQcHhGjImLURxMnNePpSZIkSZI0azMNCiLi4prVBym66PenCAgGAwe0XWntJyLWAd7IzHcpegkMbrB9Xop5Cl6obc/Md4BbgEEzOPWcwJSa9TnK830FOA+4AvguEOX2K4EdgA2AUeWwh8/IzKGZOSgzB/XrO0dznqYkSZIkSbM0qx4FG0XE1yJiI+CbwEUUXeJ3As7hs13pO6WIWAu4FDixbDoXOCUi1iu3zw0MBS7MzEkNjp0T2BJ4aAan/zdwcLnvHsBCZfvqwH8y80Fgq7qdM7NuvoNjcNiBJEmSJKkCvWax/esUk/stQtGT4CyKv35nzdct2rLANtIzIsYAfYEXgeMz8w6AzHwiIg4DLirnLJgC/An4Sc3xG5XH9wAuqTu2EccAl0XECcD1TO+RcDVwU0S8QNGLoNaV5bUOmr2nKEmSJElS8800KMjMh4GvAUTEBZn57fYoqq1lZsxi+63ArTPYNgxoePeDGZ3nKWCdmqbja5ZXr1n+Qc3yO8DfM/NTJEmSJElqZ02ezLCrhAQdWUT0Ar4DXDyrfSVJkiRJagvNueuBZiIixjR4HNXM4wdQ3B5xeGY+0DZVSpIkSZI0c7Oao0BNlJkrzObxrwMLt1I5kiRJkiS1iD0KJEmSJElSPYMCSZIkSZJUz6EHndiAZZfnlL9eU3UZkiRJkqQuxB4FkiRJkiSpnkGBJEmSJEmqZ1AgSZIkSZLqGRRIkiRJkqR6TmbYiU18/UOe+sldVZchSZI6gFVP2aLqEiRJXYQ9CiRJkiRJUj2DAkmSJEmSVM+gQJIkSZIk1TMokCRJkiRJ9QwKJEmSJElSPYMCSZIkSZJUz6CgFUXEnRGxZ4O2n0fED5tw7F4RMT4ixpSPtdusUEmSJEmSZsCgoHVdCezeoG134IomHLsg8NPMXKF8PNzaxUmSJEmSNCsGBa3rWmCbiOgLEBGDgHeA0yPi+bKnwBblttMj4k8R8XhEHEkRFLxTWeWSJEmSJGFQ0KoyczzwILBt2bQncDnwu8xcHjgWqB2G8EVgEPA7YB7gjIh4OiLOiYiejV0jIg6PiFERMeqdj95rmyciSZIkSeq2DApa35XAHuXybuV6r4j4P+B4YImafW/OzIlZ+EFmLgVsDGwEHNLYyTNzaGYOysxBC/abv82ehCRJkiSpezIoaH1/pxh+sAHwGrA6cB7FPAXfBaJm3wkNDy57JVwFrNb2pUqSJEmS9FkGBa0sM9+jGH5wLsWwg9WB/2Tmg8BWMzouIpYvv84F7FqeQ5IkSZKkdmVQ0DaupBg+cA1wNbBxRLwALDCTY46IiJeAx4AHKEIGSZIkSZLaVa+qC+iKMvOvwF/L1bcpehXU+UG5z+kNjjkROLE96pMkSZIkaUbsUSBJkiRJkuoZFEiSJEmSpHoGBZIkSZIkqZ5BgSRJkiRJqmdQIEmSJEmS6nnXg06s74B5WPWULaouQ5IkSZLUhdijQJIkSZIk1TMokCRJkiRJ9QwKJEmSJElSPYMCSZIkSZJUz8kMO7HXXnuN008/veoyJElSO/D/fElSe7FHgSRJkiRJqmdQIEmSJEmS6hkUSJIkSZKkegYFkiRJkiSpnkGBJEmSJEmqZ1DQQUTE2IhYsuo6JEmSJEndW6cNCiIiI2JMRLwcEfdHxA4Ntm9atr9S7nNORPQqt20WER9HxPMR8VJEnFJB/atHxCHtfV1JkiRJkmam0wYFwNTMXCEzlwIOBc6MiL0BImJ54G/AiZm5JLAKsAzw45rj78vM5YE1gOMjYoH2LZ9BwEbtfE1JkiRJkmaqMwcF9TLzSeDrwM/Kpu8D52TmiHL7R8ARwGERMVeDwxcDXgU+aOzcEXFpRPw6Iv4bEeMiYvOI+FdEvBYRZ9Xsd0xEPF32UhgSEb3L9rERcXJEPBURz0TEqhGxOXAusGdE/LfmcvuX53g2Ilaf/e+MJEmSJEnN0yWCAoDMfAboGxGLAGsBwxts/wB4AVixbNowIp4DHgWGZObUmZx+CWBd4DfAP4HvAF8Ejo2IecoP/gcD6wMrAYtQBBO1118VuBo4LjPvBk4Crs7MdWp265WZqwBXAMc2VkhEHB4RoyJi1Mcffzyzb4kkSZIkSc3WZYKCUi9gMtAHmNbI9gQ+LZfvz8wVgQHAIeWH/Rm5MTMTuA94PDOfzcw3gFcoeiRsD1yamR+UgcPvgS1qjr+y/DqcYgjEjFxefh0BLNXYDpk5NDMHZeagueZq2DlCkiRJkqTZ02WCgohYB3gjM9+l6CUwuMH2eSk+pL9Q256Z7wC3UMwZMCN14cJUYFJN+xSgJ0VAURtMRLlvnbpjJpf7z8jEJu4nSZIkSVKb6BJBQUSsBVwKnFg2nQucEhHrldvnBoYCF2bmpAbHzglsCTw0GyXcDhwcEfNGRE/gEIrwYWY+AfpHRMzGdSVJkiRJalWdOSjoWd4e8RWKuQOOz8wbATLzCeAw4KJy++PAM8BPao7fKCLGAE8At2TmHS0tJDNvBq6hCBueBF4ELpnFYXcDqwKjW3pdSZIkSZJaWxRD79UZLb744nn44YdXXYYkSWoHp59+etUlSJLaWUSMzsyZDZNvE525R4EkSZIkSWplvaouoCMphyLU+r/MHFJJMZIkSZIkVcCgoEZmrlB1DZIkSZIkVcmhB5IkSZIkqZ6TGXZigwYNylGjRlVdhiRJkiSpDTiZoSRJkiRJqpxBgSRJkiRJqmdQIEmSJEmS6hkUSJIkSZKket4esRN7992nuOrq9asuQ5IktYG99nyw6hIkSd2UPQokSZIkSVI9gwJJkiRJklTPoECSJEmSJNUzKJAkSZIkSfUMCiRJkiRJUr1OExRExLCIGFx1HZIkSZIkdWWVBwURcWxEPBER4yLivxXW8Z2IeC8i+lZVQ1nHkhFxfJU1SJIkSZK6r0qDgojYCPg6MCgzlwF2a6XzRgsO2w94EtipNWqYDSsA21dcgyRJkiSpm6q6R8GCwDRgCkBmjo2IJSLinoh4PiIejIglGh4UET+LiDFlL4R9yraDIuLvEfEA8JOIuDcitim3RUSMjYhFGisiIlYG+gC/pAgMaredEhHPlNfaobG2iOgRET8p256PiDPK/QZGxJiac50eEaeWy8Mi4syIeCQiXoqITSJiReAyYKPy+c0zW99dSZIkSZKaqeqg4FbgNeChiPhK2ZbAIZm5PHAP8I1GjrsxM1cAdgB+WtO+Qdl2CnA5sHPZvh7wTGa+OYM69geuBW4EvhwRCwGUIcQ2wLrAssD9jbUBBwLrAGuUj60jYscmPP9FM3Mt4EzgtMx8DtgXuC8zV8jMDxseEBGHR8SoiBj1wQdTmnAJSZIkSZKartKgIDMnZ+ZuwMnALyJiCPAGxV/ULwK2BT7XowD4JCLOAX7SYPvwzByfmQlcDWxXtu9KERx8TjlMYV/gmsz8GLgD2KvcvAvwi8yckJnTMvOdGbRtD1yYmZ+W5/gTsEUTvgVX1dUNLNOE/cnMoZk5KDMHzTtvr6YcIkmSJElSk3WIT5qZ+c+IuItijoD+wCfAOcCjwBdr942I1YArgYOA/6PokVBnQs053yy7769NERicM4PLDwaWBO4qpzboCywOXFguN/yzfWNtvSiGUNSXCUwt9+tZ0967wbGTyq+TG+wnSZIkSVIlqp7McI2IGFi3CnxKERTcAjxL0aOgoVXLbSOAzWZxicuBE4EXMvP9GeyzP/DDzFwsMxcDBgCrRcRywJ3AkRExR0T0LOc4aKzt9rKtT0TMSTHPwS0UvSMWiIiFI6JHE+qFIiRZsNxfkiRJkqR2VfWH0QWA2yNiLHAfxV/9f07RU+C/wKuNHHMbMC/wEvCFWZz/78x82EEfYHfgb3VtmfkJRY+FfSl6FTxPEUw8Aaw2g7ah5frTwEPAdZl5Z2ZOBn4I3FVe4+VZ1AvwMEVg8oKTGUqSJEmS2lsUw/m7poiYA3gOWLkMALqU5Zfvlz89Z7Wqy5AkSW1grz0frLoESVLFImJ0Zg5q7+tW3aOgrR0K3NwVQwJJkiRJktpCh5jMsC1ExNXAQKbf+YCIuAz4Us1uIzPzwHYuTZIkSZKkDqvLBgWZuWcjbftWUYskSZIkSZ1FVx96IEmSJEmSmsGgQJIkSZIk1euyQw+6gwUWWNUZkSVJkiRJrcoeBZIkSZIkqZ5BgSRJkiRJqmdQIEmSJEmS6hkUSJIkSZKkek5m2Ik9+e4HrHXNrVWXIUlSt/LIHttWXYIkSW3KHgWSJEmSJKmeQYEkSZIkSapnUCBJkiRJkuoZFEiSJEmSpHoGBZIkSZIkqZ5BgSRJkiRJqmdQ0AQRcWxEPBER4yLiv1XXI0mSJElSW+lVdQEdXURsBHwdGJSZn0TEwIpLkiRJkiSpzdijYNYWBKYBUwAyc2xE9IiIn0TEMxHxfEScARARAyNiTN2BEXF6RJxaLg+LiF9GxEsRsXpEbBARD0bECxHxp3KfrSPi4Yh4LiJ+3P5PVZIkSZLU3RkUzNqtwGvAQxHxlbLtQGAdYI3ysXVE7NiEc82VmUsDLwNXA9/KzOWA70bEgsBZwGbAqsDmEbFOwxNExOERMSoiRk354P3ZfGqSJEmSJH2WQw9mITMnA7tFxM7AL8qvCwMXZuanwKdlj4AtgMdncbrryq8bAqMz8z/lNd4qg4aVgQfKfeYGlgM+MydCZg4FhgLMtfxKObvPT5IkSZKkWgYFTZSZ/4yIu4AngdEUwxHqBDCVYnhCz5r23mVbnQnl174N2qF4Le7MzN1bs25JkiRJkprDoQezEBFr1ExgGMCnwEPAkRHRJyLmBPYDbgHeABaIiIUjogfFMILG3A8MjohVy2ssDjwIbBoRK5RtMzpWkiRJkqQ2Y4+CWVsAuC4iegMfAucAlwI/B56mCA6GZuadABHxQ+Au4AmKuQg+JzPfiIjDyvP2AW7IzGMj4jjg9oiYRhFGDGvLJyZJkiRJUkOR6TD3zmqu5VfKFc/9ddVlSJLUrTyyx7ZVlyBJ6iYiYnRmDmrv6zr0QJIkSZIk1TMokCRJkiRJ9QwKJEmSJElSPYMCSZIkSZJUz7sedGJfWGBeRjmhkiRJkiSpFdmjQJIkSZIk1TMokCRJkiRJ9SIzq65BLRQRHwLPVF2HNBv6A29XXYQ0G3wPqzPz/avOzvewOrumvIeXycyF26OYWs5R0Lk9k5mDqi5CaqmIGOV7WJ2Z72F1Zr5/1dn5HlZn15Hfww49kCRJkiRJ9QwKJEmSJElSPYOCzm1o1QVIs8n3sDo738PqzHz/qrPzPazOrsO+h53MUJIkSZIk1bNHgSRJkiRJqmdQIEmSJEmS6hkUdFARsVdEvBgRYyLikAbbVo+IRyJiXET8OiJ6lO2bRsSz5XGnVFO5VGjhe/jMiHi6PO5b1VQuFVryHq7ZPiQi7mjfiqXPauG/w70j4rcR8Wp57DLVVC+1+D28Q9n+VET8oJrKpcLM3sPl9m0j4qkGbR3iM51zFHRAETEP8CSwATAVeBhYIzPfKrffA/wUuA24C/gF8A/gWWB34HngIWCvzHy4ncuXWvQezszrI+Io4HfAQsDjwLqZ+Ur7PwN1dy19D5fbVgduAp7LzK3avXiJ2fp3+ExgUeAoICh+V5zc/s9A3d1svIdfBtYD3gFGAgdn5uPt/wzU3TXhPXwFkMCemdmrbAs6yGc6exR0TNsCwzPz1cz8H8U/flsCRMTCwLKZ+a/MnApcBnwFWBd4IzMfzcyPgGvKdqkKLXkPk5lDMnNa+Q/oKxSBgVSFFr2HI6In8GvgjGrKluo1+z0cEb2Bg4HvZebUzJxiSKAKtejfYYrfH+YB5gCmAG+3e+VSYYbv4dI3M3OfBsd0mM90BgUd01LAuJr1V4AB5fKSwEuNbJvZMVJ7a8l7uF5EbAHMTdGrQKpCS9/DPwD+BbzQ1gVKs9CS9/DSFH+F/WXZ7fWSiOjbHsVKjWjpv8M/AZ4AXgPuKT+gSVWY6eezzHyvuce0J4OCjqkPMK1mfRpFd5WZbZvZMVJ7a8l7GICIOJBi+MFXy78SSFVo9nu4DLg2An7eLhVKM9eSf4cXAZanGNL4BWA+4PA2r1RqXEv+HV6c4v27PMWHqw0iYrt2qFVqTEs+n3WYz3QGBR3T68ASNetLAi/PYtvMjpHaW0vew0TE94C9gY0y8zMTu0jtrCXv4WOAVYCngD8DG0bErW1fqtSolryH3wReyMzHMnMKcCOwcjvUKjWmJe/hwcB/MvPlzJwAXAUYFKgqLfl81mE+0xkUdEy3AttGxCIRsRjFX6huA8jMl4CPImKzcizs/sDVwAPAyhGxckT0A3YDrqumfKn57+GIWAo4ANglMx1PqKo1+z2cmbtm5nKZuQrFe/n+zNy2qiegbq8lv0u8ABARa5QzyG8H/KeS6qWWvYefoehFsEA558b2ZZtUhRm+h2eiw3ym61XFRTVzmflGeSuM+8umE4BtImL5zDwPOBD4EzA/cGlm3gsQEYcCN1B0WTk3M8d97uRSO2jJezgivgIsCzxVTPgKwE8z8w/tW73U8n+HpY5iNn6XOISiR8y8FPNt/Lm9a5dgtt7DZwGjKT7n3AJc1N61S9Ck93Bjx3zaUT7TeXtESZIkSZJUz6EHkiRJkiSpnkGBJEmSJEmqZ1AgSZIkSZLqGRRIkiRJkqR6BgWSJEmSJKmeQYEkSZIkSapnUCBJkioRERdGxEpV1yFJkj4rMrPqGiRJktpFRKwOrJ+Zf6y6FkmSOip7FEiSpO5kELBR1UVIktSRGRRIktTNRcTAiBgTEb+IiJci4o6ImDMi1oyIkRHxfETcFxGrlvufHhF/iojHI+LIiLg0In4dEf+NiHERsXlE/CsiXouIs2Zy3WERMbhcnhIRP4qIlyPinxGxQ0Q8FhFjI+JL5T6XRsRPI+LBcr8Tas51TEQ8XdY6JCJ6l+1jI+JnEfG/iNgJOBfYMyL+W24/NiKeK+s+oWzbLCKGR8RlEfFqRFweEVFu26C8/gsR8aeybeuIeLg8z4/b4jWSJKk9GRRIkiSAZYHrgIFAb2BP4Brg5MxcnuID9l9q9v8ixV/nf1euLwGsC/wG+CfwnXKfYyNiniZcvyfwMLAcsBTwLWBt4P+A42v22xD4cnmtEyJiuYjYHDgYWB9YCVgEOKLmmPcyc7HMvAE4Cbg6M9cpt40CVi7P96OI6FO2DwJOB1YolzeMiHmBq4FvZeZywHcjYkHgLGAzYFVg84ioO7ckSZ2SQYEkSQJ4LTNHZOY04F6KD+sTMnM4QGb+A1ii/LAMcHNmTszpkx3dWC7fBzyemc9m5hvAK8BiTbh+Ajdl5mSKD+//yMypwIPAkjX7/a287pvAncA6wPbApZn5QXnM74Etao65bibXfRv4EUXgMQewcNn+38x8LjM/KWtYhmLIwujM/E/5PXmrbFsZeAB4nCJoWa4Jz1eSpA6rV9UFSJKkDmFSzfJkit8RpjWyX13bhAbtn5ZfpzY41xSK3gKzMi0zpzRyjobHT65ZnhP4uJFaozxHnYa1FjtF9AfuAg4BfgtsUB4Ln/9+9AT6lvXU6gXcmZm7z+iJSZLU2dijQJIkNSaB/hGxCUA5vn9MZjb6obsd7RoRPSNiWYq/5v8buB04OCLmjYieFB/8b5nB8Z9QPK+g+Ov/u8AdFL0ABszi2vcDg2vmalicorfBphGxQtm2WcufmiRJHYM9CiRJUmOmAXsBQyJiPmAscEClFRX+RzE0YW7g6MwcD9wcEWsBD1H8xf964JIZHH83cCYwmmJOg+eBl4EbKIZJzFBmvhERhwHXlXMZ3JCZx0bEccDtETGtrGHYbD1DSZIqFtOHFkqSJHVcEXEpcEdm/rXqWiRJ6soceiBJkiRJkuoZFEiSJCKib82tAVv73EtExJgGj11aerpWLU6SJH2OQw8kSepmyi78l2bmsJq2cygmK7y4wb4LARtk5k2zec05gXmBLG9t2JJzrABcnJmb1Z0zMz+JiNOB9zLzl7NTY3nOtzOz/+yeR5KkzszJDCVJ6iYiYgFgz2bs3w+4Drg6Io4ATis39QAWBt4o1y/OzNMj4mZgIYqJEBOYn+J3jbcp7jbwEfAAcHZEHAx8p+ZyqwJPlcurAE/XbDs5M29upMQbIuLHM6n/UuCLFHc2AFiM4haHY8v1+YHHM3O/mZxjQeDwBs1XA0sB/wBeL9v6UtwisuEtJZcGvpCZY5EkqZMwKJAkqftYHdgCmAgQEYOAuh4EA4BJEXF0uX4w8Hvgusz8Tdl2UXlcf+DezFyl7sQRsSnwtwbX2wLoDdxa2xgRW2fmJRGxN1D31/ukuGNBw+UnG4QES0XE7sBtwIYUdy/YfCbPeWrNuaZSfJCvXa+r6afA/sACEVF394MbgF8BhwHnlW0HAo8DE4C/Z+ZBEbEw0+908GHNtfcG/jST2iRJ6pAMCiRJ6j7WpvhL+KIUf12/ITPXhs8PPYiInYCfA3+PiOHAbuWtCGfkA4qeA7UmAH0aaX+//LpKZg4srzcmMwdFRC+Kv/IPKgOJW8rtOwPrAR8DPwGWpeil8ACwCDC1vHUhwEWZ+ety+cTMvKM8x0HAwMw8vVzfjCIEIDN/APygHHqwZERsQxEKAPwvM39XHrNB7ROJiPWA7wPPAvtQ9Co4l6LHxcsz+X5JktRhGRRIktR97ELxYfZM4FJgv4h4uNxW16PgR8B7wEOZeUBEnA3cC0yIiPszc8PGTpyZD5Uf1E8CXi2bF6cYplDX82AximEK5zVyilnZhGI4wnjgh8BVwFcz895ZzFHwi4h4H1iNojfBUxGxVbltPuCRuh0joifTexn0YnrPg5lZAzgAWAu4B5hMEW6cmJkZ4dyLkqTOx6BAkqTuY5fM/Cgi5gCOB0Zn5mG1O0TEKGCPzBxb3pngJGAOit8ZBtTsulxEjCmXv1Mz2eF9wO3lcsOhB5s0qOfB8noAi0fEExS9BCaU7dOAf5fb+wL/oRgeMAo4IzPvncXz/RcwAtgIuBG4kOLD/wbApsBfKeYwqLMA0+cz6EXxoX9melH0GjgF2BL4L0VviT2BL0XEyFkcL0lSh2RQIElS93FgRBwDDKQYez+5pkdBnZWAmyNiMYqJ+qZl5pRySECtF2rnKKjxBtMnIvwCRchQt75C7Y6ZuRdARAykmHNgzcycSiMy85jyrgdk5scR0SsiHi83Nxx6cERmjgRGUgQEb1L0ZjiZYkLFSynmWHiWYshAnSXKfaEIOD5trJYa0ygChxEUt22cAjwJXE7RM2FxYNtZnEOSpA7HoECSpO7jbWAv4ATglswcFhHvlMtjYXqPAuAI4HfAM828xu4Uf8GH4sN5D+BL5foiwNDyOjdTfJAGWBN4AhhddtXvQ3EXhLphAQ9l5sG1FynnIPh1ea7TaXzowesUcxDsWl7rIOAmYC7g8ojYGjg+M+vuVPAliskRAfoBk2bxXKdl5qkRMTcwjmIOiDWADTPz28BjEfGDWZxDkqQOx6BAkqRuIjOvAmgwbn4scGNEDM7M92r2/UEj+87KsxQBQ91wgoMphgxcWK6vQdHDgMzcvjz/UcChwKaZOSEiegCXALdnZu3tE1viV8BRFBMtjqMYcvAHirsWTKGYV+CvEbF/2ZNhd6DuDg9zMuuggIjoA9xNMWxhLMUtHp+MiHUzc/TMjpUkqaMyKJAkqRvLzFsi4mPgw3J4QX9mPTZ/RtYFFgI2K9fXofhdo1/NPu8Dd9Ss303R02B4RLxZ7vshcEgLawCg/Cv/3RTDCfoCO1DcTvFUYHuKeRN6UEyO2D8ilqKYdLHuVoxfYPpdC9aJiLrhEwOAa+quk5mfAuuVAcdAijkP1qUYemBQIEnqlAwKJEnSaIqx+5MpJg98vXZjeUvAK4DFImIsxQfshctlKO6KMBFYvcF5Fyn3nbfB+ZYFjqYYmjCgPHYCRUhxN7AT8FBE/AP4R2aOanD8l4EhDa5TO0fBq8CfgOWAE4Gvl21zAmdlZgLbRMQAimEJ61FMSHgScEJEnELRC+HLFD0g/puZg8trX1pz3V0b3i6xQZ3HA0vPaLskSR1VFP9XSpKk7qL8sHtpZg5rwr4TM7NvG9SwJcUcA09Q3Cnhlsx8qmb7msDewByZ+d2ybQWK2ytu1gb1LJOZ48rlyFn8ghQRmwEHZeZBs9hvWLnf2FYpVJKkdmBQIEmSJEmS6vWougBJkiRJktRxGBRIkiRJkqR6BgWSJEmSJKmedz3oxPr3758DBw6sugxJkiRJUhsYPXr025m5cHtf16CgExs4cCCjRo2a9Y6SJEmSpE4nIsZVcV2HHkiSJEmSpHoGBZIkSZIkqZ5BgSRJkiRJqmdQIEmSJEmS6hkUSJIkSZKkegYFkiRJkiSpnkFBO4qIOSLiyIj4+wy2rx4Rj0TEuIj4dUT4+kiSJEmS2pUfRNvXM8A2wDwz2D4E+D6wHLAmsHM71SVJkiRJEgC9qi6gm1m7fJzacENELAwsm5n/KtcvA74CXN9+5UmSJEktlDnzx7Rps79Nak29e8Oii1ZdRYdkUNCOMvO9iJjR5iWBl2rWXwF2aPOiJEmS1DKZMHUqTJ5cPKZMafzr7LS1ZJ8Z7TtlSlHvtGnTH7Xrs7Nc95A6k0GD4D//qbqKDsmgoOPoA9T+6zoNmNpwp4g4HDgcYOmll26fyiRJkjqDTJg4ET7+uHh89FHjy83ZVrc+ceLnP4BPmdL+z7FnT+jVq/hLaO/e05dn9LVuuW/f4tiePaFHj+lfZ3e5sfWIzz9m1N6SbVJr6d+/6go6LIOCjuN1YIma9SWBlxvulJlDgaEAgwYNsv+VJEnqfCZPhgkTig/gH300fXlWX2e2re5DfXO7p/foAf36wVxzFY/a5UUXnb7et+/MP4i3pK25+9d9IJekNmZQ0EFk5ksR8VFEbAaMAPYHTqm0KEmSpFnJhA8/hNdeg1dfLb42XH733c9+sJ88uennjyg+rM8992e/zjcfLL54sVz7aOwDf8P12uU+ffwrtSQ1YFBQsYjYFVg+M88DDgT+BMwPXJqZ91ZZmyRJ6uYmToTXX595CPDqq0UA0FDdB/nFF4cll/z8B/26r4211X7t29cP8pLUziKdPbTTGjRoUI4aNarqMiRJUmc0YQK88AKMHfvZD/+1IcD48Z8/bo45YIklpocAjS0PGFB8yJckzZaIGJ2Zg9r7uvYokCRJ6oqmTSs+7L/wQvF4/vnpyy+8AG+++dn9e/SAxRYrPuwvtxwMHtx4CLDAAv6FX5K6OIMCSZKkzurjj+HFFz8bAtQtv/giTJo0fd8ePWDppWH55WGXXYowYPnlYeDAIgRYdNFisjxJUrdnUCBJktRRZcL//td4r4Dnny+21ZpnnuLD/xe+ADvtVIQBdYHA0ksXM+dLkjQLBgWSJEkdwaefwpNPwn//Cw89VDweeaSYS6BOBCy1VPHhf/vtiwCgNgxYcEGHBUiSZptBgSRJUnubMAEeffSzocDjj0+/beDcc8Paa8NBB8Eqq0wPBJZZpphMUJKkNmRQIEmS1JbGj58eBtQFA88+WwwrAOjfH774RTj++OLrF78IK6xQzCkgSVIFDAokSZJaQya88sr0UKAuGHj55en7LL10EQTssw+ss06xvMQSDheQJHUoBgWSJEnNNW0ajBnz2aEDDz0Eb79dbI+AlVcubjFY10vgi1+EhRaqtm5JkprAoECSJGlmpk6FZ56B0aOLYKAuHPjww2J7796wxhrFLQfrAoE11yzmGZAkqRMyKJAkSaozeTI89dRnQ4GHH4aPPy62zzlnMcngAQdMHzqw2mrQp0+VVUuS1KoMCiRJUvc0aVJxp4G6QGD06OJOBJMmFdvnnrsIAr7xDVh33SIYWHll6OWvT5Kkrs3/6SRJUtf3ySfTb0dY11ug9naE881XBAHHHFN8XWcdWHFF7zwgSeqWDAokSVLXMmUKPPII3Hff9FDgySeLuQagmFBw3XXhhBOKQGDddWHZZb3zgCRJJYMCSZLUuX3wATzwAIwcWTweeAA++qjYtuiiRRCwyy7Thw8stZShgCRJM2FQIEmSOpdXXikCgXvvLR6PPlrcrrBHD1hrLTjkENh44+Kx5JJVVytJUqdjUCBJkjquqVPhiSeKQKCux8C4ccW2fv1ggw3gtNOKUGCDDWCeeaqtV5KkLsCgQJIkdRwffwwPPjg9GLjvvmJoAcCAATB4MBx/fBEMrLWWdyCQJKkN+L+rJEmqzhtvTB9GMHJkMfHglCnFttVWg332KUKBwYNh4EDnFpAkqR0YFEiSpPaRCc89ByNGTJ9fYMyYYtscc8D668P3vlcEAxtuCAsuWG29kiR1UwYFkiSpbdTdpnDEiOnhwJtvFtv69y8CgSOOKL6us04RFkiSpMoZFEiSpNbx8cfw738XgcCIEXD//TBhQrFt4EDYdlvYZJNiGMEqqziMQJKkDsqgQJIktcz48cW8AnW9BUaPhsmTiwBgjTXggAOmBwPeplCSpE7DoECSJDXNuHHTewuMGAFPPlm09+kD660HJ5xQhAIbbQQLLFBtrZIkqcUMCiRJ0udNm1YEAXW9BUaMgJdfLrbNO28xr8C++xY9BtZbD/r2rbZeSZLUagwKJElScUeCZ5+FO++EO+6A4cPhnXeKbQMGFIHAiScWPQbWWAN69qy2XkmS1GYMCiRJ6q5ee60IBuoer7xStC+zDHz1q/DlLxcBwbLLOvGgJEndiEGBJEndxfvvFz0F7rijCAbq5hhYaCHYYgvYaivYcktYbjmDAUmSujGDAkmSuqpJk4pbFNYFA//5D0ydCnPOWfQWOPjgIhhYay3o0aPqaiVJUgdhUCBJUlcxbRo8/PD0YGDECPjkk2I+gfXXhx/8oOg1sMEGMMccVVcrSZI6KIMCSZI6q0x4/vnpwcBdd02fgHC11eAb3yiCgU03Le5UIEmS1AQGBZIkdSbvvAO33jo9HBg3rmhfcknYeeciGNhii+JOBZIkSS1gUCBJUkeWCY88AjffDDfdBA88UAwxmH/+IhA48cQiHFhxRScglCRJrcKgQJKkjubDD4veAjfdVAQEr71WtK+7LpxyCmy/Pay3XjH3gCRJUiszKJAkqWqZ8Oyz03sN3HMPTJ5czCuwzTZFMLDddrDYYlVXKkmSugGDAkmSqjBxIgwbVoQDN99cTEoI8IUvwLHHFuHAxhtD795VVilJkrohgwJJktrLSy9N7zVw553FrQvnnLOYa+D444twYODAqquUJEndnEGBJEltZfJkuO++6b0GHn+8aF92WTj00CIY2GyzIiyQJEnqIAwKJElqTW+/XfQYuOkmuO02eP996NULvvxlOP/8IhxYeWXvUCBJkjosgwJJkmbXhx/C9dfD5ZfD7bfDlCkwYADssUcRDGy1VTExoSRJUidgUCBJUktMnAj/+lcRDtxwQ7G+9NLFXAN77QXrrGOvAUmS1CkZFEiS1FRTpsDddxfhwHXXFcMKFl64mG9gn31gww2hR4+qq5QkSZotBgWSJM1MJjzwAPztb3DVVfDmmzDPPLDbbkU4sOWWxRwEkiRJXYS/2UiS1JjHHivCgSuugLFjYY45YMcd4etfL+Yd6Nu36golSZLahEFBO4qIvYBzganA2Zn5x5pt6wO/BeYBbgFOyMyplRQqSd3VCy8UwwouvxyeeAJ69oStt4YzzoCvftUJCSVJUrdgUNBOImIe4HxgA4qg4OGIuCEz3yp3uRA4GHgcuBbYDrixilolqVt5/fViSMHf/gYPPli0DR4Mv/0t7LlnMQeBJElSN2JQ0H62BYZn5qsAEXEXsCVwRbn9JWA+itekD/BqFUVKUrfw7rtw7bVFz4Fhw2DaNFh7bfjZz2DvvYu7F0iSJHVTBgXtZylgXM36K8CAmvUfAyOBScAtmflQYyeJiMOBwwGW9hdZSWq6Tz4pbmN42WXFbQ0nT4YVVoBTTy0mJVxllaorlCRJ6hAMCtpPH2Bazfo0iiEIRMScwJXAl4CngL9ExDcz83cNT5KZQ4GhAIMGDcq2LlqSOrWpU4seA3/9a9GD4MMPYfHF4ZhjinBg3XUhouoqJUmSOhSDgvbzOrBZzfqSwL/L5TWAtzLzUYCI+AtwBPC5oECSNAuZ8PDDRc+Byy+H114rJiHcYw/Ybz/YdNNikkJJkiQ1yqCg/dwK/DQiFgF6ABtRhAEALwIrRMQyFHMV7Aw8U0mVktRZjRtXTEj417/Ck09C796w3XZFOLDjjjDnnFVXKEmS1CkYFLSTzHwjIk4B7i+bTgC2iYjlM/O8iDgIuB3oCzwIfK+aSiWpE3nnHbj66qL3wIgRRdvgwXDhhcUdCxZaqNr6JEmSOiGDgnaUmZcCl85g283Aze1ZjyR1ShMnwo03Fj0Hbr65mJRw1VXhJz+Br38dBg6sukJJkqROzaBAktTxTZsGw4cX4cA118AHH8CAAcWkhPvtV9za0EkJJUmSWoVBgSSp43r00SIc+Nvf4NVXYZ55YLfdinBg882dlFCSJKkNGBRIkjqWl14q7lbw17/C449Dr17FpITnnw877QRzzVV1hZIkSV2aQYEkqXoffQTXXguXXALDhhVtG20Ev/0t7LUX9O9faXmSJEndiUGBJKkamXD//UU4cOWV8OGHsMIKcOaZsO++sNxyVVcoSZLULRkUSJLa12uvwZ//DJdeCs88A/36Fb0GDj64uLWhkxJKkiRVyqBAktT2Jk2CG24oeg/ccktxF4NNNoGTToI994S55666QkmSJJUMCiRJbefhh4tw4LLLYPx4WGIJ+P734aCDYMUVq65OkiRJjTAokCS1rvHji2DgkkuKoKBPH/jqV4uhBVtv7S0NJUmSOjiDAknS7Js6FW67Df74R/jnP+HTT2GddeA3v4F99oEFF6y6QkmSJDWRQYEkqeWefbboOfDnPxeTFC60EBx5ZNF7YK21qq5OkiRJLWBQIElqng8/hKuuKgKCkSOhRw/Ybjv49a9hxx2LoQaSJEnqtAwKJElN8+9/w4UXwtVXw8cfwyqrwLnnwv77w4ABVVcnSZKkVmJQIEmascmT4Zpr4Je/hAcfhHnmgX33LYYWbLABRFRdoSRJklqZQYEk6fPeeguGDoUhQ4q5B1ZcsRhacOCBRVggSZKkLsugQJI03aOPwq9+VdzecNIk2GYb+P3v4StfKeYikCRJUpdnUCBJ3d3UqXDDDUVAMGwYzDVXMbTgmGPgC1+oujpJkiS1M4MCSequ3nsP/vhH+M1v4MUXYeml4Wc/g0MPhQUXrLo6SZIkVcSgQJK6m2eeKeYbuPRS+Ogj2GQT+PnPYZddoJf/LUiSJHV3/kYoSd1BJtx2WzG84F//gj59YJ994NvfhnXWqbo6SZIkdSAGBS0UEYtm5htV1yFJM/XRR/DnP8MFF8DTT8Nii8EZZ8ARR8Cii1ZdnSRJkjogg4JmiohdgAuA3sDiEbEJsH5mnl9tZZJUY9y4Yu6Biy8u5iIYNAj+8hfYa6+iN4EkSZI0A97rqvnOBtYF3i/X7wUOr64cSSplwj33wO67w3LLwS9+UdzecORIePBB2G8/QwJJkiTNkj0KWmY8kOXyXOVDkqpz993wve/B6NHFHQtOPBGOOgqWWqrqyiRJktTJGBQ03yXAX4F5I+JbwIHA1dWWJKnbGjOmCAiuv764veFFFxU9B+Yyv5QkSVLLGBQ0U2aeFxGbA68BKwDnZua1FZclqbt57z0466xiksI55oCf/ASOOw7mnLPqyiRJktTJGRQ0U0R8A7guM+8u1xeLiK9l5hUVlyapO5gyBYYOhR/9CMaPh4MPLgKDAQOqrkySJEldhJMZNt/JmTm+biUz/wecXl05krqNW2+FtdaCb30LVl+9mI/gD38wJJAkSVKrMihovk8iYqG6lYiYF3AacUlt56mnYPvt4StfgUmT4O9/h7vugi9+serKJEmS1AU59KD5zgeGR8RQYApwMHBxtSVJ6pLGj4fTT4cLL4R+/eDnP4djjinmJJAkSZLaiEFBM2XmHyLiKWAnoC9wRmbeWHFZkrqSTz+FIUPgjDPggw/giCOK5YUXrroySZIkdQMGBS3zAPA80BMgIhbPzNeqLUlSp5cJN94IJ5wAzz0H22wD559fzEcgSZIktRODgmaKiO8AZwHvAZOBABJYrsKyJHV2jz4Kxx8Pd94JK68MN90E220HEVVXJkmSpG7GoKD5vg+sm5nPVl2IpC7gjTfgtNOKuxfMPz9ccAF885vQu3fVlUmSJKmbMihovqeAl6ouQlInN3Ei/OpX8JOfwCefFJMU/vCHsOCCVVcmSZKkbs6goPmeA0ZGxI3ApLrGzDy7upIkdRqZcO21cOKJ8OKLsOOOcN55xXADSZIkqQPoUXUBndBrwD+BaUDvmockzdzo0bDpprDnnsXtDm+/HW64wZBAkiRJHYo9CpopM8+ougZJncybb8IPfgB//GNxi8Pf/Q4OPRR6+U+wJEmSOh5/S22miNgM+BGwOMUdDwDIzJUqKklSRzV1ahEKnHoqTJgA3/1usTzffFVXJkmSJM2QQUHzDQWOA44HvgtsDfhbv6TPuu8++Na34OGHYcst4de/hlVXrboqSZIkaZaco6D5emTmTcDD5fLPgL2rLUlSh/Hmm3DwwbDxxvDWW3DVVcVcBIYEkiRJ6iTsUdB8/46IwcCNwNCIeBD4pOKaJFVtypTpwww++ghOOqlYnnvuqiuTJEmSmsUeBc2Umftm5r2ZeTdwMjAG2L4px0bEXhHxYkSMiYhDGtl+ekS8HBFjI2KjVi5dUlsZORIGDYJjjoH11oPHHoNzzjEkkCRJUqdkUNBMEfHbuuXMvDUzz6eYr2BWx80DnA8MLh9nR8TCNdsPAQYBKwHLAv9t5dIltbY33oCDDoLBg2H8eLj6arjtNlhllaorkyRJklrMoQdNFBEbAwOBXSNiZM2m/sDXKSY4nJltgeGZ+Wp5vruALYEryu3HAbtmZt0whomtVLqk1jZlClx4IZx2Gnz8MXz/+3DKKfYgkCRJUpdgUNB0CwNbAf0o7nRQ5xNgryYcvxQwrmb9FWAAQET0BhYDDomI3YEngcMyc3wr1C2pNd17b3E3g0cfha23Lu5msPLKVVclSZIktRqDgibKzOuB6yPi6cw8twWn6ANMq1mfBkwtl/sDCwB3A6cAvyy/fm5IQ0QcDhwOsPTSS7egDEkt8sYbcOKJ8Oc/w1JLwTXXwG67QUTVlUmSJEmtyjkKmm+rFh73OrBEzfqSwMvl8tvAhMy8PTMT+AfQ6J8oM3NoZg7KzEELL7xwY7tIak1TpsAFF8BKK8Hll8MPfgBPPQW7725IIEmSpC7JoKD53ijnK2iuW4FtI2KRiFgM2Ai4DSAzJ1PcdvEr5b47Av9plWoltdyIEbDuuvCd78AGGxR3Mzj7bOjXr+rKJEmSpDZjUNB8XwKGRcRrEfFsRDwXEc/O6qDMfINiOMH9wEjgBGCbiPhuucuRwGkRMYZi7oKft035kmbpf/+D/feHL38Z3nsPrr0WbrnFuQgkSZLULThHQfO1dOgBmXkpcOkMtr0AtKSngqTWMmUK/OY38KMfwcSJcPLJxcMeBJIkSepGDAqaKTPHRcTqwGYUkxHekZnPVVuVpNk2YkRxN4PHHoNtt50+L4EkSZLUzTj0oJnKuw5cCywOLE1xJ4S9q61KUovVDjN4/3247jr4178MCSRJktRt2aOg+U4ANszMdwAi4ufAcODKSquS1DxTpsCQIXDaadOHGZxyCsw1V9WVSZIkSZUyKGi+qAsJADLznYjwk4XUmdx3Hxx1FDzyCGy9dTEvgT0IJEmSJMChBy1xZ0QMjYjVI+ILEXEhcG/VRUlqgjffhEMOgY03hvHj4eqr4dZbDQkkSZKkGgYFzXcc8DrwR+CvwAfAUZVWJGnmpk6FCy8sbm/4l7/ASSfBU0/BHntARNXVSZIkSR2KQw+aKTMnAj+KiF8BH5frkjqqBx8shhmMHg2bbw6//S2sumrVVUmSJEkdlj0KmikiNouI54GHgHERcW9ErFx1XZIaGD8eDj8cNtgAXnsNLr8c7rzTkECSJEmaBYOC5hsKHJyZy2TmosAvgD9XXJOkOtOmwe9/X8w78Mc/wnHHwdNPw9e+5jADSZIkqQkMCppvSmbeU7eSmdcC81RYj6Q6o0fDhhsWPQlWWw0eegjOPx/mnbfqyiRJkqROw6Cg+f4QEWdHxAoRsUxEfBe4MSIWr3tUXaDU7bz7LnzrW7DeejBuXDFh4fDhsMYaVVcmSZIkdTpOZth8R5df92nQvmf5NYHl2q8cqRubNg3+/Gc48cRiToJjjoEzzoD556+6MkmSJKnTMihopsxctuoaJAGPPFL0Ihg5shhucNttsPbaVVclSZIkdXoGBc0UEfMDewNLAj3r2jPz5KpqkrqV99+HH/4QfvMbWHDBYsLCAw+EHo6kkiRJklqDQUHz3Qy8DjwITKm4Fqn7yITLLoPvfhfefBO++U0466wiLJAkSZLUagwKmm/RzNyo6iKkbuWJJ+Coo+Cee2D99eHGG2HQoKqrkiRJkrok++o23+8j4uSIWDEilq57VF2U1CVNmFBMVLj22vD44zB0KNx/vyGBJEmS1IbsUdB8AfwAOBCYVrYl8IXKKpK6mky4/nr4znfg5ZfhkEPg3HOhf/+qK5MkSZK6PIOC5jsWWDkzX6u6EKlLevHF4jaHN90Ea6wBl18OG29cdVWSJElSt+HQg+YbxvSeBJJay6efwtlnw2qrwbBhcN55MHq0IYEkSZLUzuxR0HwDgGci4iHg07rGzNymupKkTu7uu4vJCp9+GnbbDX75S1hqqaqrkiRJkrolg4LmO63qAqQu44034IQTitseLrtsMdxg++2rrkqSJEnq1gwKmikzh1ddg9TpTZ0KF10EJ58MH38Mp55aLM85Z9WVSZIkSd2ecxQ0QURcX7M8OSI+rXlMjohPZ3K4pFqjRsEGG8C3vgXrrguPPQY//rEhgSRJktRBGBQ0zdfqFjKzd2b2qXn0zsw+ddsjYpFqSpQ6uPfeg6OPhvXXh1degb/9De64A1ZeuerKJEmSJNUwKGiCzJzYjN3vbbNCpM4oswgFVlkFLryw6Enw9NOwzz4QUXV1kiRJkhpwjoLW5ycfqc4zzxR3M7jrLhg0qJiscN11q65KkiRJ0kzYo6D1ZdUFSJX75BM47TRYc00YPRqGDIEHHjAkkCRJkjoBexS0PnsUqHu7+eZiLoIXX4T99oPzzoNFF626KkmSJElNZI+C1ndi1QVIlXj5Zdh9d9hhB5hjjmK4wV/+YkggSZIkdTIGBc0UETtGxAMR8XJEvBYRr0fEa3XbM/PvVdYntbspU+D882HVVeFf/4Kzz4ZHHoHNN6+6MkmSJEkt4NCD5vsdcCTwIDCl4lqkaj38MBxyCDz0EOy4I1xwASy7bNVVSZIkSZoNBgXN9zpwW2ZOqroQqTKTJsGPfwznngsLLQTXXAO77ebtDiVJkqQuwKCg+U4A7ouIW4D6sCAzz6yuJKkdPfBA0YvgqafggAPgF7+ABResuipJkiRJrcQ5CprvbOA14COKoKDuIXVtH30Exx0HG20EEyYUdzf4058MCSRJkqQuxh4FzbdYZg6uugipXd19Nxx2GLzwAhx5JJxzDsw7b9VVSZIkSWoD9ihovssi4uiIGBgRi9c9qi5KahPvvw9HHAFbbAE9esCwYTBkiCGBJEmS1IXZo6D5Dii/nlDTlsByFdQitZ2bbipCgtdfh+9+F844A+aaq+qqJEmSJLUxg4Jmykzv/aau7e234dhj4bLLYLXV4LrrYP31q65KkiRJUjsxKGimiPh6Y+2Z+bf2rkVqVZlw9dVw9NHw7rvwwx/CySfDHHNUXZkkSZKkdmRQ0Hxb1yz3BjYEngEMCtR5vf46HHUUXH89rLsu3HEHrLlm1VVJkiRJqoBBQTNl5sG16xHRF7ioonKk2ZNZ3OLwuONg4kQ491w4/njo5T8NkiRJUnflp4HZNxX4QtVFSM02blwxWeGtt8LgwfCHP8BKK1VdlSRJkqSKeXvEZoqI22oedwIvAnc28di9IuLFiBgTEYfMYJ8TI2JMa9Ysfca0afDb38Lqq8O998JvfgPDhxsSSJIkSQLsUdAScwK/At4CpgAvA3vO6qCImAc4H9iAohfCwxFxQ2a+VbPPosCBbVG0BMCzz8Jhh8GIEbDNNjB0KCyzTNVVSZIkSepA7FHQfItn5jWZOTwzR2bmS8ARTThuW2B4Zr6amf8D7gK2bLDPBcDZrVyvBFOmwM9+BmutBY89BpdcArfcYkggSZIk6XMMCppvQkQMrFuJiAFANOG4pYBxNeuvAANqzrMfMB4YObOTRMThETEqIka99dZbM9tVKjz2GGy4IZx0Emy3HTz5JBx0EERT3raSJEmSuhuDguY7BbgnIv4vIn5G8cG+Kb0A+gDTatanUQxBICJWA44Cjp/VSTJzaGYOysxBCy+8cLOLVzcybRqcd15xu8Nx4+Cqq+Daa2HAgFkfK0mSJKnbco6CZsrMGyPiMeArQF9gl8x8rAmHvg5sVrO+JPDvcvlwYAngYaA3sFREPJ6Zq7dW3epmXn4ZDjwQ7r4bdt21mIugf/+qq5IkSZLUCURmVl1Dt1BOVPhf4IsUPTnuA9bIzI8a7DcQuCMzV5jVOQcNGpSjRo1qg2rVqV11VXHbw8mT4YIL4OCDHWYgSZIkdUIRMTozB7X3dR160E4y8w2KYQv3UwxXOAHYJiK+W2lh6jo++AAOOAD23htWXhkefhgOOcSQQJIkSVKzOPSgHWXmpcCls9hnLDDL3gTSZ4wcCfvvX8xF8MMfwqmnQu/eVVclSZIkqROyR4HUmU2eDKedBl/+crE+YgSccYYhgSRJkqQWs0eB1Fk99xzstx88+GBxu8Nf/QrmnbfqqiRJkiR1cvYokDqbTLj4YvjiF4uw4Oqr4ZJLDAkkSZIktQqDAqkzeftt2G03+MY3YIMN4NFHYY89qq5KkiRJUhdiUCB1FrfeCmusATffDOedB7fdBksuWXVVkiRJkroYgwKpo/vkE/jOd+ArX4EFFyzmJDjhBOjhj68kSZKk1ucnDakje+QRWG89uOAC+Pa3YdQoWGutqquSJEmS1IUZFEgd0bRpcP75sP76MH483HJLcVeDOeesujJJkiRJXZy3R5Q6mldeKW53eOedsMsu8Pvfw8ILV12VJEmSpG7CHgVSR3LNNbDmmnD//UVA8Pe/GxJIkiRJalcGBVJH8MEHcPDBsOeesMIK8PDDcNhhEFF1ZZIkSZK6GYMCqWr33Qdrrw1//jOceiqMHAkrrlh1VZIkSZK6KYMCqSpTp8KZZ8Imm0AmDB8OP/4x9O5ddWWSJEmSujEnM5Sq8MYbsO++xYSF++4Lv/0tzDdf1VVJkiRJkkGB1O6GDYN99oH33oOLL4ZDDnEuAkmSJEkdhkMPpPYydSqcdRZsuSXMOy/8+99w6KGGBJIkSZI6FHsUSO3hzTdhv/3g9tvh61+H3/0O5pmn6qokSZIk6XMMCqS2NmxYEQ68+y78/vf2IpAkSZLUoTn0QGor06ZNH2owzzzFUIPDDjMkkCRJktSh2aNAagu1Qw322QcuusihBpIkSZI6BYMCqbXdc08RDowfD0OH2otAkiRJUqfi0AOptUybBmefDZtvDv36FUMNvvENQwJJkiRJnYo9CqTW8NZbxVCD226Dr32t6EngUANJkiRJnZBBgTS7aocaXHSRvQgkSZIkdWoOPZBaato0+OlPpw81eOABOPxwQwJJkiRJnZo9CqSWeOst2H9/uPVW2HvvYqjBvPNWXZUkSZIkzTaDAqm57r23mIfg7bfhd7+zF4EkSZKkLsWhB1JTTZsG55wDm20Gc84J998PRxxhSCBJkiSpS7FHgdQUb79dDDW45RbYay/4/e8daiBJkiSpSzIokGalbqjBW2/BkCHwzW/ai0CSJElSl+XQA2lGMuHcc4uhBn37Fnc1OPJIQwJJkiRJXZo9CqTGfPghHHQQXHcd7LknXHyxQw0kSZIkdQsGBVJDzz8Pu+wCTz0F//d/cOyx9iKQJEmS1G0YFEi1br21mI+gR49ieautqq5IkiRJktqVcxRIUMxH8LOfwfbbw9JLw6hRhgSSJEmSuiV7FEgffQSHHgpXXlnc+vCPf4R+/aquSpIkSZIqYY8CdW9jx8LGG8NVV8E558AVVxgSSJIkSerW7FGg7uuuu4oeBFOnwk03wXbbVV2RJEmSJFXOHgXqfjLhl7+EbbaBRReFBx80JJAkSZKkkkGBupdPPoEDD4TjjoOddoIHHoAVV6y6KkmSJEnqMAwK1H28/DJssgn85S9w5plw7bUwzzxVVyVJkiRJHYpzFKh7uOce2GMPmDgR/vEP2HnnqiuSJEmSpA7JHgXtKCL2iogXI2JMRBzSYNuREfFERIyLiJ9UVWOXkwlDhsCWW8ICCxTzERgSSJIkSdIM2aOgnUTEPMD5wAbAVODhiLghM98qd5kGrA30AR6MiJsy875Kiu0qJk2Cb30L/vAH2GEHuOwymG++qquSJEmSpA7NHgXtZ1tgeGa+mpn/A+4CtqzbmJkXZebkzPwIeBpYuKI6u4bXXoPNNitCglNPhX/+05BAkiRJkprAHgXtZylgXM36K8CAhjtFxGrA+sChjZ0kIg4HDgdYeumlW7/KruD++2G33eDDD+Gaa2D33auuSJIkSZI6DXsUtJ8+FMML6kyjGIJQLyK+AvwT+HpmvtfYSTJzaGYOysxBCy9sp4PPufhi2HRTmGuu4taHhgSSJEmS1CwGBe3ndWCJmvUlgZfrViLia8CPgC0zc0Q719b5ffopHHUUfOMbsPnm8J//wOqrV12VJEmSJHU6BgXt51Zg24hYJCIWAzYCbgOIiDmAs4GvZObY6krspN54o7irwYUXwoknws03w4ILVl2VJEmSJHVKzlHQTjLzjYg4Bbi/bDoB2CYilgdupOhtMDoi6g75S2ae0f6VdjL/+Q/suiu88w5cfjl87WtVVyRJkiRJnZpBQTvKzEuBS2eweY72q6SL+MtfiqEGiy0G990Ha69ddUWSJEmS1Ok59ECdz7RpcNppcMABsOGGMGqUIYEkSZIktRJ7FKhzmTgRDjoIrrwSDj20mJegd++qq5IkSZKkLsOgQJ3Hm2/CV78K998P554L3/seTJ/TQZIkSZLUCgwK1Dk8+STsuCO8/jpccw3svnvVFUmSJElSl2RQoI7vjjtgjz2gb18YPhzWX7/qiiRJkiSpy3IyQ3VsF18M220HSy0F//63IYEkSZIktTGDAnVM06bBSScVtz/ccku4915YZpmqq5IkSZKkLs+hB+p4Pv4Y9t8frrsOjjwSLrgAevlWlSRJkqT24KcvdSyvvw477wyjR8MvfgHf+Y53NpAkSZKkdmRQoI7jscdghx1g/Hi4/voiMJAkSZIktSvnKFDHcMstsPHGMHUqjBhhSCBJkiRJFTEoUPWGDCl6Eiy3XHFng3XWqboiSZIkSeq2DApUnalT4bjj4Fvfgu23L+5ssOSSVVclSZIkSd2aQYGqMWEC7Lor/PKXxYSF118Pc89ddVWSJEmS1O05maHa3yuvwE47waOPwm9+U/QokCRJkiR1CAYFal8PPQQ77ggffAA33gjbbVd1RZIkSZKkGg49UPu54QbYZBPo2RNGjjQkkCRJkqQOyKBAbS+zmItgl11glVWKOxusuWbVVUmSJEmSGmFQoLY1ZQocfXRxd4OvfhWGD4cBA6quSpIkSZI0AwYFajsffFBMWjhkCHzve3DNNdCvX9VVSZIkSZJmwskM1TbGjSsmLXzqKRg6FL7xjaorkiRJkiQ1gUGB2sZ//1vcBvGWW2CrraquRpIkSZLURAYFahu77gqbbQYLLFB1JZIkSZKkZnCOArUdQwJJkiRJ6nQMCiRJkiRJUj2DAkmSJEmSVM+gQJIkSZIk1TMokCRJkiRJ9QwKJEmSJElSPYMCSZIkSZJUz6BAkiRJkiTVMyiQJEmSJEn1DAokSZIkSVK9yMyqa1ALRcRbwLjZOEV/4O1WKkcdm6919+Fr3X34Wncfvtbdh6919+Fr3X3M7mu9TGYu3FrFNJVBQTcWEaMyc1DVdajt+Vp3H77W3Yevdffha919+Fp3H77W3Udnfa0deiBJkiRJkuoZFEiSJEmSpHoGBd3b0KoLULvxte4+fK27D1/r7sPXuvvwte4+fK27j075WjtHgSRJkiRJqmePAkmSJEmSVM+gQJIkSZIk1TMo6IYiYq+IeDEixkTEIVXXo7YVEU+Ur/WYiPhj1fWo9UTEHBFxZET8vUH7dyLipYh4JiK2q6o+tZ6ZvNbv1/x8n1lVfWodEdEnIoZExLMR8VxE7F62+zPdxczktfZnuouJiB4RcXv5Wj8TEduW7f5cdzEzea075c+1cxR0MxExD/AksAEwFXgYWCMz36qyLrWdiBiTmStUXYdaX0SMBR4C5snMrcq25YF/AesCSwF3AMtk5uSq6tTsm8FrPQcwKjPXqLI2tZ6IWAwYnJnXRMRKwIPAl4Ab8Ge6S5nBa70o/kx3ORERwGKZ+XpEfAU4C9gb/6/ucmbwWm9MJ/25tkdB97MtMDwzX83M/wF3AVtWXJPalmlg17U28KsGbbsCV2Xmh5n5JDCW4hcRdW5r8/nXeiHg3fYvRW0lM/+XmdeUy88CU4Cv4c90lzOD13ox/JnucrLwerm6DPAI/l/dJc3gte60/1cbFHQ/SwHjatZfAQZUVIvaWET0AxaNiBci4u6IWK/qmtR6MvO9Rpr9Ge+CZvBazw+sFhHPR8SNEWHPoS4kIg4GHgUWxJ/pLq3mte6HP9NdUkScGBHjgeOAM/H/6i6rkdd6fjrpz7VBQffTB5hWsz6NYgiCuqDM/Cgz583M5YAhwN9ndYw6PX/Gu4nMfDIzFwJWBO4G/lRxSWolEfF94NvAvvgz3aXVvtb+THddmfmz8rU9GbgVf667rEZe66c668+1QUH38zqwRM36ksDLFdWidpSZVwN9I2L+qmtRm/JnvJvJzGnARcBqVdei2RcRvwVWATYuu7D6M91FNfJaA/5Md2WZeR0wN/5cd3k1r/VC5Xqn+7k2KOh+bgW2jYhFyol0NgJuq7gmtZGImC8iFiqXtwPemUEXZnUdNwFfi4i5ImJVim7LD1dbktpCRCxaDi8C2I9iMjR1YhGxAbByZh6UmR+Xzf5Md0GNvdb+THdNEbFc+Ts3EbEhMBF/rrukGbzWPTvrz3WvqgtQ+8rMNyLiFOD+sumEzPyoyprUphYE7igmYeV/wJ7VlqO2lpmjI+KvwBMU/0Edlt7epqtaDrgiIqYAY4BvVFyPZt/awKCIGFPTdjTgz3TXszaff63/AHzTn+kuZ37glojoCbwJ7O3/1V3W/DR4renE/1d7e0RJkiRJklTPoQeSJEmSJKmeQYEkSZIkSapnUCBJkiRJkuoZFEiSJEmSpHoGBZIkSZIkqZ5BgSRJkiRJqmdQIEmS2l1EzNlI2wYR0bcZ5/hSRKzbjP17N3VfSZK6s8jMqmuQJEldVEQsBiwJHJ2ZB9W0D8vMzWrWVweGAt8CLiybFy6/vgXckpmn1+y/OHArMAnYLDMn1GxbHlgf6Ae8BywFXAecBlyQmY+25nOUJKmr6VV1AZIkqUvbA+gD9IqIJYG/lu1rR8SwcvmHwG+B3TPzWWADgIg4DJiSmZfWnjAiFgD+CXybIky4JiK+mpkTy11eAHYH3gXmAfpm5riI2BAYFhFrAu9l5o1t8YQlSersDAokSVJbWgOYE1gH2L+uF0Ftj4KI2APYC9gpIl7JzCtndLKIWBm4GjgTGAlMAxakCAAOKIOGk4GtgS8CrwBvRcS7wLPABOBopgcWkiSpAYMCSZLUlgYARwFnAa/X9CJYOCIeLJf/DYwG9gU2jIh/ZeZ2DU8UEceW5zo4M0dGxOnAK5n5u4gYA9wSEX8AzqH4Had3+TgH+D7wZmZeHxHfAa5viycrSVJXYFAgSZLa0q7AqsCXKOZG2qxuQ0QcRPG7yBXAPcBtmTkpIuapOf60iPgmcAHwNLBeZr7f8CKZeUc5pGAJ4HTgQeBt4HlgU+B44FflhIZzZuZ7rfs0JUnqOgwKJElSW7oE2BgYB1xV06MAYDEggK8Bj8/g+B83nKNgRsoJDZ8pg4ZTgWWBJ4EPM/MHEfEkcB7wQAuehyRJ3YZBgSRJaktnUNyZ4KzMvCkiVgEey8zbanoU/ANYF9isla75Q+BeYHmKHgY/KNt/CzwCrNJK15EkqUvqUXUBkiSp68rM5xs0XQhsWC7PUeySb7XyNT+gCAduB24GzoqIHsBJFPMhHFuuS5KkRtijQJIktZvM/Dgibo6Ix4C5gd1qt0fE34H5I+IBitsqUs5R8HhmHtaUa0TEXsDXKYYY3EdxG8WVgPuB/YAhwDUNry1JkgqRmVXXIEmSupmIiGyjX0IiYn3gmdpJDyNig8x8oGZ9YGaObYvrS5LU2RkUSJLUBURE78yc3E7X6gtMy8xP2+N67S0i5i2HL7TX9Xpm5tT2up4kSbPi+DxJkjq58oP74xGxUjOPWyki1pjFPpdGxGYNmk8HDpjB/gtFxA7NqOFnEbF7zXrPiPhPRPSfRU07NvUazRERiwDj6m7RWF7roJnsv0xEbNXMaywWEVdGYSOKyRwlSeowDAokSer8TgSWA/4ZEU838lin4QHlB/G7gevqPhQ32L5ARBzenCIioh9wHbBsRBwREa+Uj9ciYnLN+unl/r2BfYERNafZFOgJTIiIvuWjZ0T0iYira/brM4PnOjki5m5O3Q3sD1yZmR/O4DkeGxGLlct9KeY66B8RgyPizYh4oOYxusHtIOtcCIwqh17cDyweEV+ZjZolSWpVTmYoSVInFhHbAHsATwOHZuaDZXsAvwDWBp5scMwcwJXAH4B+wMURsU9mTqvZbXVgC2Biecwg4OJy2wBgUkQcXa4fBLwPXA1cl5m/KdsvKo/tD9ybmfW3JYyIrwJ/Bd4GHiw/fG9AERwsADxc7roo8L1y3/Vq6vu09nw15x07g2/VLJXBxXeAbWew/dvAkcCfI2JeipDgysy8IiIGN/Ea3wfmB/4Pils+RMQhwE0RsUVmPtPS+iVJai0GBZIkdVIR8SWKD+NbAvMA10bEocATwFCKD9w7Z+bEmmP6UYQE7wNnUPz1/gbgTxFxaM28A2sDS1F8UF8MuCEz1y7PcQ4wJjMvrjnvTsDPgb9HxHBgt8wcP4uncE1mHlQePwxYEtgaWDEzJ5XtvwcmNfd700LH8P/t3XecFdX9//HXB5YOdhAUFcUu9sWuYCxYsIuxi1ERjf5M1Ng1iiVqNLav2JXEijUxtthRVETAhh3pCCsWFOnsnt8fnzPc4XK3ALs7W97Px+M+du/M3JkzM2fOnPnMmTOwRgjhi/wRZnY8/srFXUIIP5nZrXhgpMTMrgeeBZ5Nv5khBj8eS30/HTgpzmNRnwQhhI/M7FxgiJkdHUJ4vYbWT0REpEoUKBAREam/vgN+H0IYC2BmxwCv4RfWzwB90hekZrYBfhf8S7yJ/brAQuBg/IL2fTM7MYTwEXAQcBQwABgEHGtmH8VZJS0K/grMAD4MIRwfl3ENMBR/dOC9EMKOS7E++wC3AumelosoHCiosDdmM2tO1R+xDEB7vOVCIX3xbdUrhPBtHPYnYBVgNNA/DjvQ/LWOiWbAzJieK/FXNu4RQihZIgEhPGpms4AnzOz+EEJ5aREREalxChSIiIjUUyGESWb2a3z84DBgP+BBYAp+53qomT0HfAVsjPdlcB1wTWzyfgYwI4RweXwU4DzgXTO7CTgohDArPqZwNjAyfbccwMxGAIcnrxk0s4OA84EWeB2jU2ry9cxsTPz/rHJW6d6YzofN7Ij4DH9zlgwUNAfmmNkDLP44AsAa8e/rwM7lLCffBOB9fNvckFq/jsCmeKuLHUIIU1O/KQIeBX4OIfzbzDYFrgwh3Jb6/Ur49gB4AfgUeNWfCinoRmBzYMUqpltERKRGKFAgIiJST5nZfXhwYAjwP+CcEMJvcdw1wJ5AT+CPwGrAbrG1wBLiRfl1ZvY40Bo4wczOBLoAtwALUi0KEhsCL5jZ58D9wDX4axMXmll+HWNsgT4KDjGzHeKgtePfwcAJwCXAlRQOFKwI/BpCOLHANhkf16dKfQakfvc7vHPHG2Jw5E944GQ+cGFekABgIP5YRtI6YG9gpaSjxuhw4BUz2y+E8ALeceHjZtYFeDGEsElq+ZcBzUMI3+EtRURERDKjQIGIiEj9dRZwMjAV71Pg/AJ3q7sCKycBhMqEEMYBmNlmwBHAOcBLIYQ3zeyn+P/4OM2iFgVm9jc8aLE0nfE9k9dHQdK537HAIDNrBbQC5uT9bkNg8lIsp1JJvwBx++0M7Bg/F6WnM7PVgW2Af+ItEO4xswPxxxPydcE7RuyKtyhIbIP3I5HWGgUIRESkjlCgQEREpJ5KtR6g0BsA4rjxyzjvx5N5p4wHnjOzXUIIM/Kmv7DA9MskdoJ4QJxfW2A23pfC3Wa2JtA6hDBtuRdU/vJfxx9dKLQ+GwGXhRB2jC0DCCE8GztwfBg4Knm1Ygx+nBxCSB65SN5GcQbeIiFtJaBKwRwREZGapkCBiIiIVEkI4SUzmw3MjI8WrAYsWI5ZHm5mPeP/HQHifNuFEH42s3Z4nwXfhxAWAtfENy48a2ZHAJcVmOcaBYYtj3lJ2qJdgK/zJwoh/GJm/wH64I9hgD8ike5Mshn+ysoi/O0QFltQtIzzvRcREZE6QIECERGRBsDMRpczqrovnEcCs/AAwfv4Yw/5adkBf4tCx9iioQnQPtW6YSj+9oX81yOC90nwQwxIAPwnhDAhTrM13t9C9xDCl8DjBZY9Pn/YchoMPBb7a2gClOF9DywhhHCPmXU0s6TfgrHApJiu7fCOJj8FeocQSs1slJmtE+f57xDCiGpOu4iIyDJRoEBERKQBCCF0KzS8ui+cQwizgJaVTDMMfz6/XPH1ha+mftMzNa4oLqMshJDuyPBzvE+ELyuY9Q54IKNaxMcQOizF9NPwTg7zfQQckw4GhBC2AX8cIXYmKSIiUieYzksiIiIiIiIikmiSdQJEREREREREpO5QoEBEREREREREFlGgoAaY2RFmNs7MxpjZHwqM72VmX+QN62FmX8ffXVx7qRURERERERHJUR8F1Sy+yulzvDOlUrzzos1DCNPj+MeAAPQJIRTFYYa/aukw4FvgQ+CIEMJHFS1rtdVWC126dKmR9RAREREREZFsjRw58ocQQvvaXq7eelD9egFDQghTAMzsdWAP/DVRAP1DCDPMrE/qN9sCJSGET+JvngT2wYMM5erSpQsjRuhNSiIiIiIiIg2RmU3IYrl69KD6rQWkd+ZkoFPyJYQwY2l/k2Zm/cxshJmNmD59+vKnVkRERERERCRFgYLq1xwoS30vwx9BqJbfhBDuDiEUhxCK27ev9RYoIiIiIiIi0sApUFD9pgJrpr53BibVwG9EREREREREqp0CBdXvf0AvM+tgZh2BnYCXK/nNMGAjM9vIzNoAhwJP13A6RURERERERJagzgyrWQihJL7e8L046BxgbzPrGkK4oZzfzDezk4D/4o8hXBdCyKTTChEREREREWncFCioASGEQcCgSqYpyvv+ErBhzaVKREREREREpHJ69EBEREREREREFlGgQGrE1KnQowdMm5Z1SkRERERERGRpKFAgNWLAAHj7bf8rIiIiIiIi9YcCBVKtWrUCM7jzTggB7rjDv7dqlXXKREREREREpCoUKJBqNXYsHH10LjBgBn36wLhx2aZLREREREREqkaBAqlWnTrBCivAvHnQvLm3KvjoI+jYMeuUiYiIiIiISFUoUCDVrqQE+veH4cOhe3f45hsYPDjrVImIiIiIiEhVWAgh6zTIMiouLg4jRozIOhkVWrgQdt0VvvwSPv0UOnfOOkUiIiIiIiL1g5mNDCEU1/Zy1aJAalRRETz4ICxYAH37QllZ1ikSERERERGRiihQIDVu/fXh5pvhtdfglluyTo2IiIiIiIhURIECqRUnnQQHHggXXgijR2edGhERERERESmPAgVSK8zgnntgxRXhmGP8rQgiIiIiIiJS9yhQILWmQwe4/3745BO49NKsUyMiIiIiIiKFKFAgtWr//eHUU+GGG2DIkKxTIyIiIiIiIvkUKJBad+ON3sHh8cfDjBlZp0ZERERERETSFCiQWtemDTz0EEyZAmeckXVqREREREREJE2BAsnEdtvBZZfBww/D4MFZp0ZEREREREQSChRIZi66CHbYAfr3h8mTs06NiIiIiIiIgAIFkqGiInjwQViwAPr2hbKyrFMkIiIiIiIiChRIptZfH266CV57DW69NevUiIiIiIiIiAIFkrmTT4YDD4QLLoDRo7NOjYiIiIiISOOmQIFkzgzuuQdWXBGOOQbmzcs6RSIiIiIiIo2XAgVSJ3ToAPfdB598ApdemnVqREREREREGi8FCqTO6N0bTj0VbrgBhgzJOjUiIiIiIiKNkwIFUqfceKN3cHj88fDLL1mnRkREREREpPFRoEDqlDZt4KGHYMoUOOOMrFMjIiIiIiLS+ChQIHXOdtt5PwUPPQSDB2edGhERERERkcZFgQKpky6+GLbfHvr3h8mTs06NiIiIiIhI46FAgdRJRUXw4IMwfz707QtlZVmnSEREREREpHFQoEDqrA02gJtvhtdeg1tvzTo1IiIiIiIijYMCBVKnnXwyHHggXHABjB6ddWpEREREREQaPgUKpE4zg3vugRVXhGOOgXnzsk6RiIiIiIhIw6ZAgdR5HTrAfffBJ5/42xBERERERESk5ihQIPVC797Qrx/ccAMMGZJ1akRERERERBouBQqk3rjxRujaFY4/Hn75JevUiIiIiIiINEwKFEi90bYtPPQQTJkCZ5yRdWpEREREREQaJgUKpF7Zfnvvp+Chh+DOO6FHD5g2LetUiYiIiIiINBwKFEi9c/HFHjA46yx4+20YMCDrFImIiIiIiDQcChRIvdOuHbz/PsyfDyHAHXf4axRbtco6ZSIiIiIiIvWfAgU1wMyOMLNxZjbGzP6QN66bmX1sZhPM7DYzaxKH7x+Hf2FmF2aT8vph7Fg4+mho0SI3rFUr+Mtf4Ndfs0uXiIiIiIhIQ6BAQTUzs3bAjcAu8XONmbVPTTIQuABYD9gCODAOvxPoBWwJHGpm3Wot0fVMp06wwgqwYAG0bOmtCVZeGa68EtZaCy64AKZOzTqVIiIiIiIi9ZMCBdWvFzAkhDAlhDANeB3YAyAGDNYNIbwYQigFHgb2ib+bDLQDWgALgR9qPeX1SEkJ9O8Pw4bBaad5nwUffAD77AN//zt06QKnnAJffZV1SkVEREREROqXoqwT0ACtBUxIfZ8MdIr/dwYm5o3bP/5/NfAZMA8YGIMMUo6nn879f/vtuf8HD4YxY+Af/4AHHoD77oODD4bzzoMddqj1ZIqIiIiIiNQ7alFQ/ZoDZanvZUBpRePMbA3gJqArHlTYwcz2LTRzM+tnZiPMbMT06dOrPfENwfrrw8CBMGGCvyHhzTdhxx39VYrPPw9lZZXOQkREREREpNFSoKD6TQXWTH3vDEyqZNwuwAchhEkhhN+Ax4GCgYIQwt0hhOIQQnH79u0LTSJRhw7eb8HEiXDzzTBuHPTuDVtsAf/6l781QURERERERBanQEH1+x/Qy8w6mFlHYCfgZYAQwkRglpn1NLOmwHHAE8BXeCuClc2sGbBfHCbVoG1bOOss+PZbePBBaNIETjgBunaFm26CmTOzTqGIiIiIiEjdoUBBNQshlAAXA+8B7wDnAHub2blxkhOA24DxwFshhKEhhI+Bq4CRwLfAFOCuWk56g9esGRx7LHz8Mbzwgj+icPbZsPba/ohCSUnWKRQREREREcmehRCyToMso+Li4jBixIisk1GvDR8O11/vnSM2bw59+8K553oQAfw1i0ce6Z0kduyYaVJFRERERKSRMbORIYTi2l6uWhRIo7bddvDkk/4axRNOgEGDYMMNoU8ff93ilVfC0KEwYEDWKRUREREREakdalFQj6lFQfWbNg1uuw2uuabw+JYtYc6c2k2TiIiIiIg0TmpRIFIHdOwIV18NX38NW2+dG96kCey5p785QUREREREpCFToECkgA02gO239wBBURGUlcGrr8JRR8Fbb2WdOhERERERkZqjQIFIOUpKoH9/GDEC+vWDLbaAL7+EHj1gjz287wIREREREZGGRn0U1GPqo6D2zZkDd90F117rgYQ994QrroCddso6ZSIiIiIi0tCojwKReqBVK/jTn2DsWLjxRvj4Y9h5Z9hnH3j//axTJyIiIiIisvwUKBBZBq1bw9lne+eG118PI0fCDjvAfvvB8OFZp05ERERERGTZKVAgshzatIG//MUDBtde60GC7beH3r29bwMREREREZH6RoECkWrQti2cf74HDK65Bt57D7p3hwMPhFGjsk6diIiIiIhI1SlQIFKN2rWDCy/0gMFVV/mbEbbdFg4+GD76KOvUiYiIiIiIVE6BApEasMIKcPHFHjC44gp4803Yems49FD45JOsUyciIiIiIlI+BQpEatCKK8Jll8H48fDXv8Jrr8GWW8Lhh8Onn8LUqdCjB0yblnVKRUREREREnAIFIrVgpZXg8ss9YHDppfDyy7DFFrDLLvD22zBgQMYJFBERERERiSyEkHUaZBkVFxeHEepav15q2RLmzSs8fM6c2k+PiIiIiIjUPWY2MoRQXNvLVYsCkQyMGwdHHw2tWvl3M/+7yir+1oTvv88ubSIiIiIi0rgpUCCSgU6dvMPDefO8FYEZ7LMPbLKJd4K41lpw3HHw/vugRj8iIiIiIlKbFCgQyUhJCfTvD8OG+d9WreDVV+Hzz6FfP/jPf2CHHaB7dxg0SI8kiIiIiIhI7VAfBfWY+iho2GbOhAcfhNtv9+DBKqvAySfDaadBly5Zp05ERERERGqa+igQkcW0awennw6jR8Prr8Puu8ONN8J668GBB/qbE8rKsk6liIiIiIg0NAoUiNRxZh4kePJJ7wTxoou874JevbxPg1tugV9+yTqVIiIiIiLSUChQIFKPrLUWXHUVTJwIDz3kjyP86U+w5prez8Gnn2adQhERERERqe8UKBCph1q0gGOOgffegxEj4Igj4J//hC22gB494IknYMECn3bqVB82bVq2aRYRERERkfpBgQKRem7bbeH++2HyZLj+em9tcMQR3uHhgAFw4YUwdKj/LyIiIiIiUhm99aAe01sPpJDSUnjxRTjooMKdHbZsqVctioiIiIjUB3rrgYhUi6ZNoXdvb2HQuzcUFeXGde0Kb72VXdpERERERKTuU6BApIHq1Ak6d/ZWBS1a+LBx42CnnaBfP5gwIdv0iYiIiIhI3aRAgUgDVlLib0N4/304/XTYe2849VTv+HCDDXzcxIlZp1JEREREROoS9VFQj6mPAllWkybB3/4G997r3085xTs97Nw523SJiIiIiEiO+igQkVqz1lowcCB88w2ceCLcfbf3X3DmmfDdd1mnTkREREREsqRAgUgjts46cNddHjA4/ni4805Ybz046yyYOjXr1ImIiIiISBYUKKiEmW1lZodmnQ6RmtSlC9xzD3z9NRxzDNx+uwcM/vxnmDYt69SJiIiIiEhtUqCgAmZ2BXAbcHP83sPMHs80USI1aN114b774Kuv4Mgj4bbbPGBw7rnw/fdZp05ERERERGqDAgUVOxLoCcwCCCEMAbbOMkEitaFrV3jgAfjyS+jTB266yYMI550H06dnnToREREREalJChRUbDbQBggAZtYVaJppikRq0frr+6sUv/gCDj0UbrzRAwYXXAA//ODTTJ0KPXroEQURERERkYZCgYKKnQe8DHQysyeA94DLsk2SSO3bcEN48EH47DM48EC4/noPGFx0EVx8MQwdCgMGZJ1KERERERGpDhZCyDoNdZqZrQzsBBQBw0MIdaYv+OLi4jBixIiskyGN0Oefw+abQ1nZkuNatoQ5c2o/TSIiIiIiDY2ZjQwhFNf2ctWioAJm9gDQJITwfAjhP3HYPzJOlkjmNt0UJk+G/faDpqmHcVZfHW64AWbPzi5tIiIiIiKyfBQoqNiuIYQfky+xNcEBGaZHpM7o1AnWXhtCgBYtwMwDBGec4eNOPRWGDfPxIiIiIiJSfyhQULEfzWyb5IuZbQwsyDA9InVKSQn07w/vvw+nnQZ77AFvvAEHHwwPPQQ77uitD667Dr77LuvUioiIiIhIVaiPggqY2Q7A48AIoBTYGTg5hPBCJb87Argu/uaaEML9qXHdgIeBlYBngbNCCGVm1gy4GTgYmA/0DCFMqGg56qNA6rKZM+GJJ/w1i0OHQpMmsM8+cOKJcMAB3gpBRERERETKl1UfBQoUVMLM2uGdGbYE3gshfF+F6T8HdsADBR8Bm4cQpsfxbwF/w9+m8DpwUwjh32Y2AFgdOB0wfN9U2HpBgQKpL775BgYN8lctTpkCq6wCRx/tQYOtt/bHFkREREREZHHqzLDu2gy/+98G2NPMjq5k+l7AkBDClBDCNDwYsAeAmbUH1g0hvBhCKMVbFuwTWxOcCPwlhFAaQlhYWZBApD7ZYAO4+mqYMAFeegn23hvuuQe23Ra22gpuvhmmT886lSIiIiIiAgoUVMjMHgb+hXdguFf87FnJz9YC0o8MTAY6xf87AxMLjFsb+Am42cy+NrMHzKxlOWnqZ2YjzGzEdF1ZST3TtCn06gWPPgpTp8LAgf4Iwp//DGusAYccAs8+CwsUJhMRERERyYwCBRXrAWwRQjg2hHBi/Pyhkt80B9Jvly/DH0GoaFwHoCtwE7ApsCLQr9DMQwh3hxCKQwjF7du3X+oVEqkrVl7ZO0AcPhxGj4azzoJ334WDDoLOneHcc304eFChRw+YNi3bNIuIiIiINAYKFFTsTWCNpfzNVGDN1PfOwKRKxn0PjA0hfBpCWAg8B2y0LAkWqY822wxuuAEmT/YWBTvvDLfcAptvDt27w+9/D2+/DRdfrNctioiIiIjUNHVmWAEzexPYFhgFzEuGhxD2ruA3q8fpt8YDMe/inRnOiuM/Bc4E3sb7L7gYeAf4GDgG+AwYDDwfQhhUUfrUmaE0ZNOn++MICxcWHr/11tChA7RvX/HfNm2q1lni1Klw5JEweDB07Fi96yIiIiIisiyy6sywqLYXWM/8dWl/EEIoMbOLgffioHOAvc2sawjhBuAE4J94B4mDQghDAczsD3h/CCsAL8b/RRqt9u1h4kQ45xx45hmYOxeaNfOOEbt1g1mz4Pvv4auv/O/s2YXn07Jl4QBC/rBbb/XXOF52Gdx1V/W/iUGBCBERERGpL9SioBJmVgS0x19ZCEAI4bvsUpSjFgXSGJx2Gtx9NzRvDvPnw6mneieI+WbN8lYI06d74KCyv3PnVr7s1VeH1q29VULySX+vaFz+96uvhocfLj/9IiIiIiL51KKgDjKzs/FWBc2BH/A3FEwC1s0yXSKNSUkJ9O8P/fp5wGDq1MLTJRfkXbpUPs8Qci0SvvgCrr8ehg3zQESzZrDRRrDTTt6qYNasxT8//7z499mz/XdVdccd/mnRomrBChERERGR2qZAQcXOBNYD7gROx99M0DfLBIk0Nk8/nfv/9turZ55m0Latf9ZbD557zh87aNnSL/p33XXp7vovWOABgyRwkA4kTJoE990HI0b4vM08UDFvHmy6Key/v3923tmDFCIiIiIiWVOgoGILQgg/mtlnwFbAa8BTQP9MUyUi1aqqrRbK06wZrLiifwr54ANvsZAEIo46CrbbzgMUt9zib3xYcUXo1cuDBvvu6/0miIiIiIhkQX0UVMDMzgI+AH7E31DwE/4aw4MyTVikPgpE6odDD4VOnRYPRCQtJWbOhFdfheef98+0ad7qYPvtc60Nttqq+jtXFBEREZG6L6s+ChQoqCIzWxXoAnwYQijLODmAAgUiDU1ZGXz4oQcMnnvOWyIArLkm7LefBw323NP7YhARERGRhi+rQEGT2l5gfWJmi67CQwg/hhBGAsMzTJKINGBNmsC22/orGocP99YFDzwAO+4Ijz0GBx8Mq64K++wDt90GY8cu/vupU6FHD/+diIiIiMiyUouCAszsRGAn4HDgidSo9sAmIYSNM0lYHrUoEGk85s/3DheTRxS++sqHb7JJ7hGFxx6De+7RKxhFREREGgo9elCHmNlmQHfgWuD81Kg5wBshhOmZJCyPAgUijdeYMbmgwSuvFJ6mZUuYM6d20yUiIiIi1UeBgjrIzPYPITyfdTrKo0CBiAB88w2cdBK88473c9CiBRx+uL9NoWPHrFMnIiIiIstKfRTUTfdknQARkcpssAFstpn/bwbz5kFRkYIEIiIiIrJsFCio2INmdoGZbWhmaySfrBMlIpKvpAT69/d+Cpo29bcmLFiQdapEREREpD4qyjoBddwR8e+pqWEBWC+DtIiIlOvpp3P/l5bC0UfDOefArbdmlyYRERERqZ8UKKhACGHdrNMgIrK0jjoKPvgAbroJuneH447LOkUiIiIiUp/o0YMKmFlTMzvdzJ40s8Fm1jfrNImIVMX110PPntCvH4walXVqRERERKQ+UaCgYrcAuwN3A/cD+5rZ37JNkohI5YqKYPBgWG01OPRQ+OGHrFMkIiIiIvWFAgUV6wUcEUJ4OYTwP+Bo4OBskyQiUjUdOnjfBdOm+eMICxdmnSIRERERqQ8UKKjYQqB16ntroFlGaRERWWrdu8Mdd8Crr8LFF2edGhERERGpD9SZYcUGAkPMbBBQBpwI3JNpikREltKJJ3rnhtdfD8XF0KdP1ikSERERkbpMgYIKhBBuM7NPgf3wbXVZCOHFjJMlIrLUbr4ZPv7YgwabbALdumWdIhERERGpq/ToQeXGA8OAl+NfEZF6p3lzeOIJaNcODjkEZszIOkUiIiIiUlcpUFABMzsbeAc4CDgJ+NTMDso2VSIiy2aNNeDJJ2H8eDjmGCgryzpFIiIiIlIXKVBQsbOArUIIJ4QQ+gC7AtdnnCYRkWW2885wyy3wwgtwxRVZp0ZERERE6iIFCio2DZiR+j4BmJVNUkREqsdpp0HfvjBgADz7bNapEREREZG6Rp0ZVuw14H9m9giwADgEGG5mRycThBAeySpxIiLLwsxfmfjpp3DccTB8OGy0UdapEhEREZG6Qi0KKtYJb0WwM9AT+BloAewVP3tmljIRkeXQsiU8/bR3cnjIITBzZtYpEhEREZG6Qi0KKhBCODHrNIiI1JS114bBg2Gvvfy1iU884a0NRERERKRxU4uCCphZdzN71MzeNrN3k0/W6RIRqS6/+x1cfz089RRcd13WqRERERGRukAtCio2GLgJGA4szDgtIiI14uyzYcQIuPhi2GYb2HvvrFMkIiIiIllSoKBivwH/F0IIWSdERKSmmMG998Jnn8GRR8LIkbDuulmnSkRERESyokcPKnYB8F8zO8XMjk8+WSdKRKS6tWkDzzwDIXjnhrNnZ50iEREREcmKAgUVOxVYA+gO7Bg/O2SaIhGRGtK1KzzyCHzyCfTr50EDEREREWl89OhBxbYG1g0hlGadEBGR2rDvvjBgAFx6KXTvDmedlXWKRERERKS2qUVBxe4F9jXTC8NEpPG46CI46CA45xwYMiTr1IiIiIhIbVOgoGKXAs8CC81svpktMLP5WSdKRKQmNWkC//oXrL8+HHEETJ6cdYpEREREpDYpUFCBEEKzEEKTEELTEELz+L151ukSEalpK6zgnRvOng2HHQbz5mWdIhERERGpLQoU5DGz9VP/71Tok2X6RERqyyabeMuC4cPhjDOyTo2IiIiI1BZ1Zriks4HT4/9XFxgfgN/VXnJERLJzyCHeZ8E118CGG8Jzz8HgwdCxY9YpExEREZGaYkHvv1pmZnZACOG/WS2/uLg4jBgxIqvFi0gjUVoKvXvD//7n3/v3h4EDs02TiIiISGNgZiNDCMW1vVw9erB8bsw6ASIiNa1tW3jpJQjBP3fcAWbQqlXWKRMRERGRmqBAwfIp+NpEMzvCzMaZ2Rgz+0PeuG5m9rGZTTCz28ysSd74gWb2ak0mWkRkaYwdC0cfDS1b5oatsALcdBOUlWWXLhERERGpGQoULJ8lntsws3Z4S4Nd4ucaM2ufmmQgcAGwHrAFcGDqt92A/WsywSIiS6tTJw8MzJ/vwQIzaNoUTjsNNt8cHn9cAQMRERGRhkSBguVTqEVBL2BICGFKCGEa8DqwB0AMGKwbQngxhFAKPAzsE8c1BW4DrqiVlIuILIWSEu+bYNgwDxD06AGPPuoBgt//HrbcEp56SgEDERERkYZAbz2ohJm1AToCTZNhIYSv4789CvxkLWBC6vtkoFP8vzMwMW9c0oLgQuBFYGwl6ekH9ANYe+21q7QOIiLL6+mnc//ffnvu/z59/C0IAwbA4YfDFlvA5ZfDwQd7ywMRERERqX/UoqACZnYlMA34L/BU/DyZjA8hfFfgZ82B9D21MqC0onFm9jtgJ+DvlaUphHB3CKE4hFDcvn37yiYXEalRTZt6/wWffQYPPghz5sChh8I228B//uOdH4qIiIhI/aJAQcVOBLqGEDYNIWweP1tU8pupwJqp752BSZWMOxPYGPgC+Bewo5n9rzpWQESkNjRtCsceC59/Dv/8J8yc6a0Kiovhv/9VwEBERESkPlGgoGKfAO2W8jf/A3qZWQcz64i3FHgZIIQwEZhlZj1jnwTHAU+EEA4JIawXQtgYOB54L4TQq/pWQ0SkdhQVwfHHw5dfwgMPwIwZcOCBsN128PzzChiIiIiI1AcKFFTseuBDM3vHzF5PPhX9IIRQAlwMvAe8A5wD7G1m58ZJTsA7LRwPvBVCGFpjqRcRyUhREfTt6wGD++6DH36A3r1hhx3gpZcUMBARERGpyyyotlYuMxsH3A68S66fAUII72eWqJTi4uIwYsSIrJMhIlKp+fP9kYSrroKJEz1gcMUVsNde6vRQREREpDxmNjKEUFzby1WLgor9FEK4IYTwbgjh/eSTdaJEROqb5s3hlFPgm2/gzjthyhTo1Qt22QVefTXXwmDqVH/14rRp2aZXREREpDFToKBi75jZk2bW18yOTj5ZJ0pEpL5q3hxOPdUDBgMHeuuCvfaC3XaDN96AK6+EoUP9dYsiIiIikg09elABM3ugwOAQQvhDrSemAD16ICL13bx5cO+9cMYZhce3bOmvXBQRERFpjLJ69KCothdYzwwLIdyVdSJERBqqFi3gj3+E/faDPn1g5MjFx+28s7cu6N7dP6utll1aRURERBoLBQoq9kczuz+EsCDrhIiINGTrruuBgA8/9DcmLFgAXbp4XwWXX57rw2C99fxVi927+99ttoHWrbNMuYiIiEjDo0BBxUYBH5jZC8BvycAQwjXZJUlEpGEqKYH+/aFfP7j7bu/Y8OmnYeZMGDUKhg/3z7vvwmOP+W+aNoVu3RYPHmy2mQcbyjN1Khx5JAweDB071s66iYiIiNQn6qOgAmb210LDQwhX1HZaClEfBSLSWJWUwAcf5IIHw4fDzz/7uFatYNttc4GD7bbzFgvJaxhPPx3uuss7VRw4MLt1EBEREalMVn0UKFBQBWbWFigLIczOOi1pChSIiLgQ4NtvFw8ejBoFc+f6+FVX9UBCWdmSv1WHiSIiIlJXqTPDOsjMNgIeAtaI38cAx4cQJmSaMBERWYwZrL++f446yoctWACjR+eCB++8A19+ueRvmzWDDTaADh2gfXv/m/4//Xe11Xz6iujRBhEREanv1KKgAmY2FLg+hPBs/H4AcEYIoVe2KXNqUSAisnROPhnuv9/7MFi4EIqLYYcd4PvvYfr03N8ffoDS0sLzWHnl8gMJHTrAww/Dc895fwt6tEFERESWhx49qIPM7JsQwgZ5w74KIWyUVZrSFCgQEVk6hx4KnTot2WFivrIyf1QhHTyo6O+PPxZ+rAH0aIOIiIgsOz16UDeVmNkuIYShAGa2C/BrxmkSEZFllA4K3H57+dM1aeL9Gqy6Kmy8ceXzLS2FL76ACy6AV1+FefN8eFERnH22BwpatVq+tIuIiIjUliZZJ6COOx24z8xGmNlw4F/AaRmnSURE6pjkNY1rreV9I7Rs6f0mrLkmXHMNbLghDBpU/uMMIiIiInWJAgUV+wq4CngaeA54ANg70xSJiEidVVLifRMMGwannQbbbANDhsAaa8CJJ8LWW8NLL/lbGkRERETqKvVRUAEzexloBrwDLEiGhxCuyCxRKeqjQESkfggBnnwSLrzQX+O4xx5w/fUeSBAREREpj/ooqJs2CiGsk3UiRESkfjODPn3goIPgrrtgwADYdls4+mi46ipYd92sUygiIiKSo0cPKvaUme2UdSJERKRhaN4czjzTWxVcfDE884x3lnjOOfDTT1mnTkRERMQpUFCxA4C3zGyCmX1tZt+Y2ddZJ0pEROq3FVbwlgTffAPHHQc33wxdu8Lf/w5z52adOhEREWnsFCio2J5AV2A3YK/4fa9MUyQiIg3GmmvCvffCxx/DzjvDeef5GxL+9S8oK8s6dSIiItJYKVBQgRDChEKfrNMlIiINS7du8Nxz8PrrsPrqcMIJ3tHhyy9nnTIRERFpjBQoEBERqSN23x3efx8efRR+/RV69YK994YPP8w6ZSIiItKYKFAgIiJShzRpAkceCV98ATfdBCNH+hsSjj8eJsQ2bVOnQo8eMG1atmkVERGRhkmBAhERkTqoRQv405/8DQnnnw9PPAEbbQR/+QtccgkMHeqvWRQRERGpbhZCyDoNsoyKi4vDiBEjsk6GiIjUgkmTYN11obR0yXEtW8KcObWfJhEREalZZjYyhFBc28tViwIREZF6YK21PFiw777QtOmS404/3VsdfP99NukTERGRhqMo6wSIiIhI1XTqBOusAyF4K4J582D77WHlleHBB+GOO3y6zTaDnj29c8QePWC11TJNtoiIiNQzalEgIiJSj5SUQP/+MGwYnHaaBw9eeAF+/tmHXXstdO4MgwbB4YdD+/awxRZw1lnwzDPw009Zr4GIiIjUdeqjoB5THwUiIlKeBQtgxAh44w14803v/HDOHDDzwMHuu3urg9128xYJaVOn+psXBg+Gjh2zSL2IiIhAdn0UKFBQjylQICIiVTV/PnzwQS5w8M47MHeuBw622soDB7vvDrvuChdeCHfdBaeeCgMHZp1yERGRxkuBAllqChSIiMiymjcP3n/fgwZvvAHvvefDCtFbFURERLKhtx6IiIhIrWnRwh87uOwyDxTMmAFPPgndukGTvNrBxhvD+efDSy/BzJmZJFdERERqkQIFIiIiQsuWcNhhsMsuue9msO220LYt3HSTv5px5ZVhxx398YSXX4ZZs7JNt4iIiFQ/BQpERERkkfy3Kqy9Nrz9tr9V4ZVXvGWBGdxwA/TqBSutBDvvDJdcAq++CrNnZ70GIiIisrzUR0E9pj4KREQkK7/95h0iJn0cjBgBpaXQrBlsv33urQo77gitWi35e71ZQUREpHLqo0BERETqjbZtvUXB3/7mrQ9+/hleeAH+/GfvFPHqq2GPPbzFQY8ecPnlHlSYO9d/f+WV/srGAQNqLo1Tp/qyp02ruWWIiIg0RGpRUI+pRYGIiNRVv/zigYDkdYyjRkFFVY7mzWHkSGjTJvdp1WrJjhWXxumn6zWPIiJSv+n1iLLUFCgQEZH6YsYM7+vgv/+FJ57w71XRqlUucNC69eKBhPK+X3QRLFy45Lz0mkcREalvsgoUFNX2AkVERKTxWWklOOAA/zRtCnff7f0ZzJ/vw/r18zcozJ7tf5NP/vdkWEnJktMkjzXka9LE+00YNKg211hERKT+UqBAREREalXyZoV+/TxgMHUq7L//8s+3tDQXNDjnHHj0UQ8SlJbCe+/B5pt7vwpHHAEHHggrrLD8yxQREWmI9OhBDTCzI4DrgFLgmhDC/alx3YCHgZWAZ4GzQghlZjYAOAJoAdwQQri9suXo0QMREZHCDj0UOnXyYMRdd8Hnn8M22/hjD5MnQ4sWsO++HjTo3Rvatcs6xSIiIktSHwUNhJm1Az4HdsADBR8Bm4cQpsfxbwF/A14GXgduCiH828xOB+4EVgVGA9uGECZXtCwFCkRERJZOWZm/peHxxz1o8N133nfBfvt50GD//f2NDiIiInWBXo/YcPQChoQQpoQQpuHBgD0AzKw9sG4I4cUQQinesmAfgBDCwBBCWQwoTMYDBiIiIlKNmjSBnXaCm2+GSZO8g8VTToF334Ujj4QOHaBPHw8izJqVdWpFRESyoUBB9VsLmJD6PhnoFP/vDEwsZxwAZvY7oC3eqkBERERqSJMmsMsucOut/jjCkCFw4onw1lveuqBDB/j97+Gpp7zvAxERkcZCgYLq1xwoS30vwx9BqGwcZnYC/vjBwbHFwRLMrJ+ZjTCzEdOnT6/WhIuIiDRWTZvCbrvB7bf74wivvw7HHw9vvAGHH+5Bg6OPhn//O/d2halToUcPmDYt06SLiIhUOwUKqt9UYM3U987ApMrGmdlfgN8DO4UQvihv5iGEu0MIxSGE4vbt21drwkVERMSDBrvvDnfc4UGDV1+FY46Bl1+GQw6B9u3h2GPhD3+AoUPhiitqLi0KRoiISBbUmWE1M7PVgVHA1ngg5l28M8NZcfynwJnA23j/BRfjjyq8AGwTQlhQ1WWpM0MREZHas3ChtzDYd19/5WIh66wDbdpA69b+N/3JH1aVaS68EO67D049FQYOrN31FRGR7OmtBw2ImfUFLo1fz41/u4YQbjCzbYB/4q9HHBRCuNTM9gGeBNL3C/4WQrivouUoUCAiIlL7pk6Fs8+GZ56BefOgqAjWXx+23x5C8E4QZ83yfg2S/9Pfk0cXlkXLljBnTvWti4iI1G1ZBQqKanuBjUEIYRAwqJxxo4DN84a9hHdgKCIiInVcp06w0kqwYIFfuM+f748qVPWOf2mpBw3SgYT8oMJ338Ejj8DHH3tLBoCuXeHJJ2tstURERBZRoEBERERkKZWUQP/+0K8f3H23tzKoqqZNoV07/1Tkm2/gww+hRQtvuTB2rL/a8Zxz4C9/gRVWWL51EBERKY8ePajH9OiBiIhIw3Xood56IQlGjBkDq64Kjz7qHSr+9a8+rlmzrFMqIiI1RX0UyFJToEBERKTxGTHCWxS8+SZssAFccw0cdhiYZZ0yERGpblkFCvR6RBEREZF6pLgYXn8dnn8emjeHPn38kYShQ7NOmYiINBQKFIiIiIjUM2aw337e2eF998HEibDrrnDwwfDll1mnTkRE6jsFCkRERETqqaZN4Q9/8I4Pr77aWxp06+YdLS5NB4siIiJpChSIiIiI1HOtW8NFF8G338If/+itDNZf3zs8nDkz69SJiEh9o0CBiIiISAPRvj3ccgt88QX07g0DBnjA4I47YMGCrFMnIiL1hQIFIiIiIg3M+uvD4MEwbBhsvDGcfro/kvDMM6AXXomISGUUKBARERFpoLbf3l+j+Oyz3p/BoYfCLrvAu+/6+KlToUcPmDYt02SKiEgdo0CBiIiISANmBgccAJ98AnffDePGwc47e9Dg7LP9tYoDBmSdShERqUssqP1ZvVVcXBxGjBiRdTJERESkHpk1C1ZaCRYuXHJcy5YwZ06tJ0lERMphZiNDCMW1vVy1KBARERFpRNq0gYkTvUVBUVFueFER7L67vzFBr1YUEWnciiqfREREREQakk6doEMHKCvzVgTz5sGGG8Lo0fDiiz7NttvC/vv7p7gYmuj2kohIo6EiX0RERKQRKimB/v39zQinnQYbbQQTJsDHH8M113gA4aqrvEPETp3gxBPhySfh11+zTrmIiNQ09VFQj6mPAhEREalJP/4IL70Ezz/vLQ1mzPBHFHbbzVsa9O7tLRFERKRmqI8CEREREalTVl0VjjkGHnkEpk+Ht96Cc87x1gjnnOOtEDbYAP70J3jlFX+EIZ9ewSgiUv8oUCAiIiIilSoqgl13hWuv9b4Mxo2D22/3QMGdd8Lee8Nqq3kniekOEa+8Uq9gFBGpb/ToQT2mRw9ERESkLpg9G15/HZ57zh9TmDy5/Gn1CkYRkarTowciIiIiUi+1bu39Fdx5p7968eOP4fzzvYVBoqjIWxuMG5ddOkVEpGoUKBARERGRamMGW2zhjygcfrh/b9oUFi6EZ5+Fu++G337LOpUiIlIRBQpEREREpEaUlPirF0eOhKOOgg4d4K9/9X4N7r7bgwciIlL3KFAgIiIiIjXi6ae9w8Mtt/Q3J0yZAu+8A+utB6eeCptv7q0M1GWWiEjdokCBiIiIiNSanXbytyA88wyUlcFBB/nrE99/P+uUiYhIQoECEREREalVZnDwwf6axTvugK+/hh12gCOOgDFjsk6diIgoUCAiIiIimWjWDPr39+DAX/8KL7wAm2wCZ54J33+fdepERBovBQpEREREJFNt28Lll3vA4OSTvZXB+uvD1VfD7NlZp05EpPFRoEBERERE6oSOHT1IMHo07LEHXHKJvyHhvvv0hgQRkdqkQIGIiIiI1Ckbb+ydHb79Nqy9trcy2HJLeO45vSFBRKQ2KFAgIiIiInXSLrvAu+/Ck0/C/PlwwAGw++7wwQdZp0xEpGFToEBERERE6iwzOOww+Pxz+L//87/bbQdHHgnffgtTp/rrFadNyzqlIiINhwIFIiIiIlLnNWsGf/yjd3h46aXw3//6GxJ69YKhQ2HAgJpbtoIRItLYWNCDXvVWcXFxGDFiRNbJEBEREal1LVvCvHlLDjeD3XaDNm3807p17v+qfE8Pa9nS53f66XDXXXDqqTBwYO2vq4g0XmY2MoRQXOvLVaCg/lKgQERERBqrqVPh3HPh6adh7lxo2hTWWMNfq1haCrNm5T6zZ/vfuXOrZ9nNm/v8ioqqZ34iIuXJKlCg4k1ERERE6p1OnWCFFbyTw5Yt/W/v3hXf8S8t9aBBEjjIDyTkf582DV56yR93KC3NzWf+fFhxRdhmG+8vIfl06eItEERE6jsFCkRERESkXiopgf79oV8/uPtub2VQkaZNoV07/1TVaafBN9/kghFHHgn77w/Dh/tn4ED4xz982lVXXTxw0L07tG+/7OsnIpIVPXpQj+nRAxEREZGadeih3nohHYx4+unc+AULYPToXOBg+HB/M0NZmY/v0mXx4ME223j/B/mmTvUgxODB0LFj9a9HTc9fRGqG+iiQpaZAgYiIiEjd89tvMGrU4sGDCRN8XJMmsNlmiwcPNtsMzjqrZjtMVIeMIvWTAgWy1BQoEBEREakfvv8ePvhg8eDBTz+VP32TJv7qx+X1v//lWjekFRXB/fdDhw7+eETyt0WLZV+WWi2IVD8FCmSpKVAgIiIiUj+FAOPG+YX8Lbd4PwhlZR4gWGklWHttaNZs+Zczfz5MmgQzZuQCBma+/EJWWGHxwEFFf1dbzd8AkajpVgsKREhjpEBBA2JmRwDXAaXANSGE+1PjugEPAysBzwJnhRDKzKwHcA/QDLg3hHB1ZctRoEBERESk/jvtNO//oHlzv7Cv7gvt/Pn36wd/+xtMn+4tHaryN/3Wh7SVVoJffikceCgqgmuu8T4Z0p/WrQt/b9my4rdG1MbjE7URjGgI/VFoHWqPAgUNhJm1Az4HdsADBR8Bm4cQpsfxbwF/A14GXgduAv4DfA0cBnwLfAgcEUL4qKJlKVAgIiIiUv9V1mFi1vMvK/MWCUngID+IMGECvP++/59cWjRpUviRh4o0aZILGqSDCR98UP7jE9ddV3kAIvm/efPKX19ZG8GIml6G1qHuLKM6KFDQQJjZ4cDBIYRj4/dHgGdDCI+ZWXtgVAhhrTiuH7ANcC9wcwhhlzj8GuDXEMK1FS1LgQIRERERqQsKtYq47TaYPRtmzcr9TT4Vfc8fN2OGP5oxY0YuEFHR4xPladq0/KDC668XDkY0beqv4KwOd95ZuGVGdS2jpudfG8vIch1atoQ5c6pnGdUpq0BBUW0vsBFYC5iQ+j4Z6BT/7wxMzBu3fzm/2ajQzGNwoR/A2muvXT0pFhERERFZDiUlfiGXbrXQtCm0a+ef5ZUEIlq0yAUibr21akGIqkyz8cYwcaK/sSLRvLkHER57bPnTD97/w6xZnv6aWEZNz782lpHFOrRuDYccAjfcUD3zbygUKKh+zYF0PLIMfwShonEV/WYxIYS7gbvBWxRUT5JFRERERJZd+lGG22+v/vkXCkQUFcGKK/qnOuS3ijjppOpvkl7Ty9A6LP0y5s714EFd7qcgC02yTkADNBVYM/W9MzCpknEV/UZEREREpFF7+mkPQGy5pf+tzj4cEkkwYtgw/zttWv1bhtah7iyjvlMfBdXMzFYHRgFb44GYd/HODGfF8Z8CZwJv450ZXgwMB8YBv8MfO/gA2DeEMGGJBaSojwIREREREZGGS30UNBAhhBIzuxh4Lw46B9jbzLqGEG4ATgD+ib8ecVAIYSiAmZ0E/Bd/DOG6yoIEIiIiIiIiIjVBgYIaEEIYBAwqZ9woYPMCw18CNqzRhImIiIiIiIhUQn0UiIiIiIiIiMgiChSIiIiIiIiIyCIKFIiIiIiIiIjIIgoUiIiIiIiIiMgiChSIiIiIiIiIyCIKFIiIiIiIiIjIIhZCyDoNsozMbDowIet0VGA14ActI/P518YyGsI61MYytA6NZxlah8azDK1D3VhGQ1iH2liG1qHxLEPrUHeWsbzWCSG0r+2FKlAgNcbMRoQQirWMbOdfG8toCOtQG8vQOjSeZWgdGs8ytA51YxkNYR1qYxlah8azDK1D3VlGfaVHD0RERERERERkEQUKRERERERERGQRBQqkJt2tZdSJ+dfGMhrCOtTGMrQOjWcZWofGswytQ91YRkNYh9pYhtah8SxD61B3llEvqY8CEREREREREVlELQpEREREREREZBEFCqRKzGy8mXUuMHyQmR2bRZoqYmb9zOz3Zvamme1SlWlrK21xmfub2Z9rc5lZKy8P1RVVySvSMJjZtmZ2fdbpSJhZTzN7Net0LC8z62JmY+L/S1XGNZRt0NCZ2cL4d6mOoXTekIbFzC43s12zTkdtKq/umz4u4na5pPZTl638/NDQ84eZ9TWze6s4bZ28ZqqIAgX1nJkFM7ujwPD7luekbGbdzOwPy5e6cue9xPMuZtbJzB4ys8lmNsHM3o3DLzezH83s2zg8mNlYMxtjZm+Xt4wQwt0hhMEVpOGmqk67tGKh8UtM4xgzO9fMrjKzBWY228y+Bb4IIdwUp7/NzErjuo2Jf98zs/3z5tsjDp9sZpPM7FozK4rjeibzNrOJZnZxBelLljOpJpdTxW31mpn1yRv2dzO7rAq/PSLmjWQ7b1XOdH8ys89i/hm1POldHjGtv5hZyypMW2NpNrPOZnZ2dc6zCsu8PJXvJpjZM2a2ylL8/g4z23A5lr8w9f804OYQwnn546own85mNqjA8C5mNj+VFx8zs7PMbEZF+7tQpWFpKh1VTPPFZrZaJdMUmdn3ZnZhNS3zNaB1UsbFYX83s3FWxWCcmW0X80zn+L2VmT1rZktVbylv/1o1V+LNrHVq/y+I6zrGlqOCbNUQvEzKWDP70MzuisOqWsbuYmYjzM+5/zaztsm4EMLIEMJ5S1GGDwWaFljGZWb2cVzWUq9vumywXBCjSkEnixd18fgdY2YXxfJp/6XNa1YgEL6UZUt5N2MW3ciojvyQN++kTP4+llX5dYHFgkF526cs2T4hhMtDCOXWx+JvK7xRYGbHxrRMMa9jjDGz9yqYfrFtUWhbW8X1ygWxzJ5gZreb2Ypx3HFxu0yqbH8AF5ufyxblvxDCSOCFquS/1DyrFESrZH8sll8rOy7Kya/5dcMQt1HBumJ5QgiXA5OSdcrPH4XSVmBd8/PDNDObU9X8sTTidrwj9X1LM1tofi7+MG6rhWb2SHUsr94JIehTjz/AQmAs0DY1bCVgDDBmOebbF7g39X080LnAdIOAY5dy3iHv+4rAV8CpQNM4bL3493Lgkvh/M6AUGAW0qOKy3gR2KbTdanCf5G+7J4F7gAnAVGBroF0c1xSYBMxP0hT36abASOD3cVhXYAqwa/zeBngU+Fv83hN4NbU9fwRWLi/PpP6vseUUWO4SeQjoBzyWN2wssCEQYj6eBLwH7J833Y1xnpPjNNcCRal0zo7j5gF/jcO7LG1eKWdaq8I03YA/pL7Pi+s0Na7X26k8vhBYM37fCRget0Ep8Kc4fhDlHGtAF6p4vKf3YSXTnQXMAFoWGLcq8ATwZVyfv1Qyr8uB0mTbAbcC/6jKNgUuBlZLfT8F2CT1/am4vb6N+WD9SvL8tGTb549bmg/QDihLbf+Jefv7A+BdoE8F8xgEHMvix1VfUuVHHNY65pkxwAJgXPx/12U57gpMsz/wDvDZUm6DbYC3gG+A74AjYrrKO65DecdYsg2AQ4BdgVdi/iuYduCm1P+dgbML7fO8fX8wfpxPjvOemMqfPwLfA2Vx3Hjg9CrkgSuWZZuXM7/F1oOlKJMqmGc/4IWYH6cBzeO+2LAKvz0aP9Yt7rv3848XPAAws8C+3jBv2GRgXN6wP+LH7lfALpWtL7lzwhjgbeBS4Pb0Po95eRRVKONSv+sK/ISXt8mxdWLefpicWvba5aTtX/l5EHgGP2abp4Z/BOxQUZ6Jx0OIv02W+33+9q9kvS4FBgJ/jb8fix+nX+QdIz3xc+yiukCBebWJ26hV/nFVxbTkr1+XuH7n48ffr8AaMS0/xbSUuw/z80qBfBnw43ixeiVwX5z/OKAE2B04N+aZFng5OA0/l+TXV9L5rwT4olAayJVllxPrr6n9cXvePPfHzxPlnr8L5b/0/shL19sFfr9Yugrtj7zp2uF5NMl/q5KqK+Ll/b2F0pW3fyutk1SWj/LS3BL4AXgvNf5W4IKlyYsFllEMfJn6fi4wCy+X1o3bewoF6hZx+r7knbMrWNYgKrlmogr1y+r8VLa8WkuIPjW2gxfiJ4JTUsP+DNyRPkiBM/FK/bdx+mZx+HjgoljgfQVsghecJcAvwKjUdBfGeXwNdIvDB+EV3auAa1LLexg4tJw0h/i3C7kT/ky8YE1OQhsBbwA/A9/GYVvEgmtmLKg2AVbGTzCP4xWWhXjFY14s4H7AezOdgF84zscrgiFui9+Aa4BLgA3i9wHAx3jFP7lgbgs8hp9on8ErXeVVeBcVGsB2wGd4QGA8XqhdnUyDnzhmx/SEuB3LUtvgN+BQPNBwZt5yVgCm4xcRPckVphsBnxBPjnm/2ZNUoAa/6PpX3D43xO09Bdgpjj8y7u9S4La4nKFxvz0b98USy4m/7QC8GKf9J35C6RzHPRK3/9i4/zaN44uBEcDr5E5+v4vjvwOGAKPjdvuFXACjYEAD6B331aKABrAmfnHzLZ5Xkgv0N5N9Clwflz0BOCq1X58BhuF5Ziiwdxxncf92KCcfbITnvVLgP3nb6TU8387AKw298bz3Yxx+Q/xbEv8uAN6Mv90lzrdLTO+bwCtx3IyY1vn4xfypeL5eGD9jgAPxCsE3wJV56Sr3Qhc/ps5NrfsSFee86S8nBgri9/2Bp1Nl2PXEEzVLllXj8QuopEyYA4yI03bH88k4/HhaFdgKv6AZD9wfpyuN+3ICXiZ8gld6RsXfT8IriCV4Xp2IlyFz4/I+wfP6B/H7bDwAGGLauuD5+Nc4z8fj/jsPP6Z+i/vjc/yY+DKmqSzOb2ZMVzM83zwf5/NTXtrGxH2ZLHNMKq89CTwXx98QhycXKeOAvhXsn8fwcuYzYOvU8PF4pemLuOzdU/vz9rgdpgH/wMuhbeJ0q8Z1fiBOv0fcDiFu03H4cb1fXO+f47p+gZ9THsDPS+MpP1CQDgD0JO+igrxAAR7Y/AH4T1z21LhPXsKDjpfg580ZMT3T47ptH9fpJjxfpM9Ti/ZB3rIXpTtO8zpejj6Dl1VdCs0zrkdZaj5fAv9NnWvviL/bHw84DyNXaW9SznZaFc9j5+EBg3PxMvaROPxH/FibiOehpKz+AM+HE4GH8Pw6g8UvOt7Ey/BS4MGY3k/xMvoRcoHa8fix8XOc/2jgNHJl8ptULVCQfzG4IX5eaJIqSx4A/k4lgQJSleOYJ2aSq2s0Z/GLngrTFacJePmSvmmTlLOziIEHvHx/paI8k9q+k/Hyu1UqHUsTKNgQP0bvS9IVt//5ecdGTzwPbgRMiMNaxWm/jPnsd/gxemVq3V4A9iJ18UOunB4LvBaHbR3zQVJON8GPgen4cXE5ngcHA8fhZURSPr+bWp9T8DpUX7w8fJdc/kzWZS9yF7nzKHwTLQkU3Ao8HMc9ClyGlwdjiYEC4Ji4D77Fy+vf43WfQXh+nouXsQ/EbfJn/LiaQ6yrpLbXlJiuZ4A2cfgDwAVUHijYJW/YOuTK/4J5glyH9UsbKBiAH79JvfTmmB+eiesxH3ia3HltGLBx3LYPxW3zJ2K+x8va7/G6zkZ42TIXz5tl+HmjYFmWl+Y98TL5u/j9jpie/nH/Jefpd/Fz2vNxuT/FeV9JLn8sqvPgdZjpwBrx+ysx7Un97ThiELCc3/fF8+5/yeXHZuXsk0H4Ta338eP7nNR6vgW8jNfJWwF3xuV8A5wap3sd2C3+fytwW/y/V1zu+ixZ/+lC+eegh4nHabn5r6oFjj518xMPkm7kKs4WD5btyBUiu+MF7gr4BeuTwBlx3Hjgovj/VcDd8f++LNmiICnwBqQOoEF4oGAz4NM4rFk8WJa4ExnHh/i3C17BGAochl8EHh3T+Bl+EXM5flFWFDN5wAuK34Dv43yej8NOw4MdpXjhezteCM2K22gkXmkqjvPpgN8VfRyvJJ6HnyDuivM9GXg5/n9NattshhdwFQUKZuAn2BJyF0TjgS3xE9RJeIH4Mn5yKov7crPU9klaUKyFX9BuUWBZ78d59ozr+Q1+cuxfTtqaJusev78A7IhfdN8Wl3Mg8HEcvz0eZR8e1+VIcoGN+eUtJ/72IWKkN6avjFzlOSnoeuOF+EH4yeOfwDl4YVYax78Zp70tLrMlHjh5GT+Zf4kXvCuRFziJ2/Bl/KS0X5zPGsTIMH4R/lGc90w8H+2On7S+A+4CxqfWZ2HcfwPxC9r/i/t1IJ4nCwbb8GPr57hOpXgB3Q04Kv7+bjzvrYYfT9PjfhyCX+CXxXU4Cc/Dv+LH3ldxXwyJ6XoTP/6/jL+ZiR9LP8Xpnojpmh2XNw+/Y1+EV/BH4ieOkpjeI0gFNvC8/W9yx2Bf/GQ1Ea9gjccrVz3x43U2uQux5E5tSUzfJXH6ENM4J853YUzv53Ha0jhuOn4hPCfum0vxPB/iur+On9C/jcv5Ca+8rByn+Q6vuM3EK2xlMd1l+An7C7zyk+yj/4vjfiVXwRuPX1i9HZeRLPum+H8psXVQ6m/Ay9x3UumdTy5oWRbTVQZ8iFewfsPz0eUxbc/gFaxSFg8UjI/rPS1ul3XwQEkpnqcmxuFJfl+Yty/vxe8ezYnT/xS3QXIRugAPLjUBegBj428vj9vrR7yM+BTf50maxsRhL8dpS/BjNuCV7F/JBQ1mxr8TU9tnYfx9cqH1eVxe0iplemo73opXBpNt/0+84pnMPwD34xXJBeQCNC/iefBPcf6X4Oe2+/EyZjaelw6Mv3kkpmUuXu4+mJr/d3hZOjV+FsT5Hxv35WS84voP/EJiWEzDmLjeH8XhST4bE/fLl3j58XHcNsPjvmgW07BW3B6PAoeUUw5b3L8n4fv8K7yM3Q0vL5LPRnEZo/ELsu/xc08RXsEP+LFfmirTv8Yr6/OBO+LwUXi5MgUv/86P8/85btO5eNmTDjqNIy9QQOEK+RIXRHj+OS1VH5oGHIDXK16Pv58Q1/vD+P/DcVuOiWn6Ke6z+XFYxzivVfEgwvy4Db/GW6UsxOsEC8iVswE/dsbH6T+P4/4Tt8ec+LtSPE8siMMeIHeH+i/xd9/i+XJo3F7JOfDNOI+t8eNhXtzGV8b98TZen5mOB9qSMuwzYBU83/wa5/8duWNoRlxWlzjsp9S4D/Dz8xxy5dXt5AIjSavIBTE90/Fj5jFygY5fyJVbj+KthrrE9X4BvzD6a9w/b8XfjcTP4VPx1g5JuX4Ano/n48HNIjzwFeI0U/C6QBme565M5ZXkJloSKNgdL8O+ien+Kc5zfFyny1LrdWrcdz/hF4UvxTR8hAcTSmMaSvA8+Gpcj0F4mRTwY/1zvC7xj/jbhfgF4XjKv5grievYM863ddz2SZkXyB2XXfCA6MK4zy6J4/9DLo+PjON/jMtLLvQX4sH3OTHtSaBgPF42nobnx6S11/i4zZ/H83QSDPmKXBn9G36MjYtpnhHTfWlcblLeX4fnhdK4vY9NlTNJoKBpXLd5eNn0LV5mnYmXV6/j5choPJ93w8vJ14E/xOV9iuePIjzPbxPn/Rh+/dGSXL0yqQePxo+Xkji/xX6P58fv8ZYHRfh574RyyuNB+HHcEr8GmYK3dOkZt9P6eJl9BX5+Nrwc+hq/WXoBuVbWQ4itK/Ay4ATgFnL17lXi39eALeP/1+LHQZe43XejkhYF6qOgAQghjAZmm1kxfudmNJ5pE/sBg0IIv4YQSvEK2+9S45Pn84fgFc3yPBr/vo0fpOk0fAYEM1svzvv1EMLcKiT/O7ywmosfdOvgd/YXhBCejdPMJnd3vRQ/Qd8HlJrZCngBWoZf2M/DC/Kkmf8veMF0B34Afo1Hu8EPlEfjvMErAN/jJ1pYfHvsiZ8ck3X9sJL1ejKEsDF+UH6bGp5cZGyNB27a4CcPUvMmtR0XxPVuHtcxX3IyBy8wNgA6AX8ws92XmNj3fwB6x23XFS88W8b13wKPHneIP5mCVyrXwyOcq+EVwRK8MlVwOdHe+AU0IYQ38X2dKDKzfwBnx3U8HN8XvfH82B0vIM/GWwCAb7sF+HbbEr+jvRawM95cvw9eAd0gTr8jflLugVf+bzSzgTHtO5k/r9sLP+muiZ88n8IvZP6OX9ydBKwZ13EXvKK8ftw+bYB947I2Bc7AL8T/HEJ4A68gPwFsi1ckZuOV/Flx2w3GK2DD8DwwKqa5N34R8lJczz5xW3wFzAsh/BjHHY2fvBfgebV9TMuc+Lvf4rqdCPwtjluIB3sWxnWdG+cxOqZpM7zi/gCe36YDu5nZqql9tzVe0ZuFH4ff4s3zVsXzyEox/Vfhkf2X4nYyPB//Fqf5JjXPp/BKXbu4H5rg5cGQmMYvgRkhhN74ifoIvDKVWAu/QD4LP36Tyslp5J6LHo/fiSVulx+A42O6/oUf62viZcY04PS4bY+I22YWvt9/wSuzd8V57Yrnzdn4xWRJavmlcZpD8QvqJnHbN43ztvh5HD8+WuPHzfQQwhfxt//CA2tfxd+mz9sd8JP+Bfh+OBQv75rg+alLnP8+lO/wuL5r45WGHfCKax/8+J8EHBRCGAI0s1x/B8/g2/u5uP475s33fTxfgOfNpJzYh1yZZXiQbhZ+x38Eng/Aj8Ef8Hz1l5i2JvE53OfxcqwlXj79gN95bI4HNh+L4y+Pf9/A9+9M/Lh7Dg8YEte1S/z/Z2BzPP83AVbHj7vvgDtDCOvj+2ML/G5tt7jtVsUrul/G330ft8kt+MXZb/h+PSluZ4DZcX4D8DKtJV5OhBDC+iGEmXG6NiGELfFzTrsQQhl+zuoGvGJmX+LlX7IO+XrG9PckV369iFdou+LH4poxf/0Q98ECvHI+KW63pH+QS1g8/3WK28qAY8xsIn4cfolfCLTAK+hr4vvY8PPLH/CyoyDz/kuuimneBNjdzLYBmprZl/Hzf3Hy6XE64vw/wPfzpvix8WhMz51x2WuRa310M35B9DWe1yfG+b0T1/Mm/CI1CXK0x/NPUzwI9CF+zCdlUbf4uyZxmMVtneSBF8mVAa3wMnafuH3nAKuGEDbF88l5cZ7NgF3NbCy5vHMPXi6uHud5GH7e2yqu81Z4oKBrTH9z/E74J3F+H5ErF9vG9d8stQs+wMvhMvz89Xe8LPsePx7GxemexsuAlfH618F4HeF5/Bw+K/6+OV6GvsmSeXUgfvOmNM5rh7gPEi/gF7lbp9IDvk++x4+dVfDj/M9xm3yIb3vw8yRmZvi58IHUvLvhZdEg/KZBW3L1DfCy4L94frkW33cr4/Xs7vH7xvj51WL6TonTbImXad3J1RcN3ycH4MfAg/j+m0osy0MIG8Zt3C/+pgt+/LbHy57uIYTZeP76DT/GwMvGcfh5aRX8nN4Hv4gGz3ODY5o/wvf/VDyPX46XX+Bl50L84pe4XTvh1xHnx+Vtgdd/Vsfrhvvg5ck9cVhb/Lwf8H3/Cn4MroWfP8HrS93j/2V4Gf87co8y3Gp5fYTE+usQPAgwAD8eHsTz+5v4vnsHP/dMi9tuI7wMuREv+9bD61yj4/j14uxfxs9Ru+L76zdgpXi+a47ffF0tzq/Q718JIUwMISTbudjMOlqq36LUqjwcQpgbQkiCG1vH4Z+EEMbEO4X7AbcG9yNeR+oZt+VuZrY6fpz8amYd8Hrwq3id66TYj8YMM2uH110Hx3PF78mVIz+EEN5K7kyWR4GChuMOvGA5Nf6fVsTiF5lGrgILfnENXjloSvnmVjLdo3gk+VByQYXKzMMPuF1T822JF1Rp6XXYEy8MSA1bGH/XnFwlFHxdb8FP8u3wyktXgBDCcPwktDZ+sknuQBXaHq3z5tusiuuXrFvaHfgBvzFeeclXhm/HfkBJCOFn/AS/S3qieKG/Dn5xvEgI4Se8kC4uJ03J/Pcnd/dwHn5h9DhwfQihk3kHhu/gJ5PSuC7Jyfe3KiynFYvvxxYx3fvgJ+XHyD0Ltjd+4dgcv3BpHdN5bmqZxP8XkAqcxEL0cWJrDAoHTorxoENvPG/ugVcSk+3/XPw7Gr/gSgIF4+Iy98MvXGekgm3b4Se05vhJ/hkKB9t2we+kdIzfy8j1rdCSXB4eiB+/LfEAxnkxXVvF8enCvD25FgyG3yFrhR8n8/CLp2QbNovDA36SC/iJdm+8Qr0wbo8+eAVxDH4yeTem+VX8YjkxJITwHvE4wlsk9MArlk3xCPsH5AJgyYVYwO/inRC33+apeY7Fo97Jhcpn+AVqclHahMXLLOI6fRmHbx3nvSVeaW4a094Hv0Anbqt3yN3hCjGAlcw/4JWFFck1KU3uiCUtA5LtmF8Gtop/O8R1aRfn0zwOX4AfXz+k5pfcVf4pbrvkjtiKeeuZPLJCnD45HlaKyx2MV1TXIVcBmAeMjheVc1m88pvvWPyieUd8X3fAK4KvxN9tRa5in9xpBw/mPoRX9lbH74CunprvKDzfd47rllz4nI3ngaS1xqp4hWc6XsFuiW/bi/F8viJ+R+7t+PtkGSF10bwusE2sCO0Wt4XhlUmL82mP75syfJv+N5XWgOeBi/ELzB74vn47bst5wBwzuzbOvwXwP3w/zsX34Sj8IiipQO+HHwO/xaDxjnjT9j/HZSYV5iHkWvsBiy5qiGlPAs0l+D4npu3LEMLG8bNOSHUcmefYuO5Hx3kZfm65Ab/QGkguTzVLpWuBmW2MB06fjtt7dExfMX7BNotcvw4L8ADaWPyC4i94ZX0KuYDYvLg93sSDTukAZNpOFK6Ql6bW+Yw47fdAj7jNDC/fk4DlYHIXIJfid60Xxt/8Eucd4l/wldwRv1AHD3D+JW6/ZB/NIHfxdBa+/+bE6TuTe6RlCLnn5Evx/NWbXOvB9vgFoMV8DDF4ErdPUfz8gp+LxpIrU7vhx2gJXt6tET8fhhCmhBCmxHSMwsvS1WLa3sUvlLchdzPgt/i9OblgQde4jOQRzq3J3bR4nNy56CG8VVxr/GL6P6n0rYHXdZriAfspQM8CefUFcheXGxIf4UuNvx8vx5+O2y0py5I07ISX9Ybn5fn4Pgv4RfOUCm6i7Rh/c0z8vEcuoAl+TirDb061if+X4nWHEjwY+AXeciMJ4j5B7s73eDyf/L84vzlx+ln4/twUz6//JpblhS7m8DxwDB7Y/iAO3x34JV7kJTeAbsJvkoW4LW8nV1fdFQ9qzccf9VuA76Mr8VZZneJ0v5BreUpcr6I4n5F4vjofL7tHx/TdiO/He+M6Gp4/Z8Rt1BE/ryetaWbjgcB34rBP43J/ib9vj+fH9Pkk8Vqc/5F4QOCfeP7siLfeTYIT88m1hroar3OcDLyUKkM6hxCejPP9H14v3yvuj/Vi+o7Cy53JyfLL+X26rtsKmBNCmBaDvuuHEI5MjV9i2vj/b6nh5V23jcKDgr3wc/QbMc1t47E/GC8TjsKPxybEc1D8rJs6B6WXVy4FChqOp/DKzbphyd5nXwFONLMVzKwpHsl8qZL5zQFWS1VYquIxPFK6I35wVtVtePRy4/j9C/wubo/4vV0cthqeZ1fFT2BjQgi/4RcCk/AT2EL8hAN+Ykzu2icF/irEuw9mtlKczwT8oEtaVhQyHC9kMLPtyF28VebV+JsBqWGTYrrWwk8qc/J+E/CDfD9y0eDr8B52u8f5tcWbCt8RQpiX/rGZtcJPihW1elgfPxmNxKPpT+CR8Bvicv6AV7TaxXQ8wuIXdlVZzvt4BB8zOxzfb+CViA9ioGZPfL8Mx4MC3+Enw+Fx2j1T8+uEF3hJ4OTgOO/W+B3uT8kLnJjZ5vi2fgmv2M7H89FL+F2UXnHSJLhQip8Qv8YvEpJofXKBuGjWcdpH4/zHhxB+oXAQ7Ti8CeOk+JtOeMW3LZ4vi+NvhuAVtLlxmvHk7lIkF/ktzGxlPEgxGa8EFuEXSgE/OXeP22ABubt4SZCnGX4S7YRfOLfCL3IMz9PzyAU29sQrenvhFxuJ+QDx2FuAHxfX4cdeGbkgTjqYlq50NI3LXZC3nZJmq0meAM+nc+Jn5VSZsGpM/5oxLZ/ifagEvCzsg1f6WpG78H4Nr2w2xysqK5lZ0ppoSzxAkuy71cn1/XAwXuFphV/4rBCnSVqugDcNbxHnvTZeOZpArl+JMrxiZHgeX4hXzonDkguTpLns6qm0FeOVmGT7Jdu1B/48+8Zx3QenKgDpvLoAaBfL8rJUmd4Mr+BvjZedfwwhdMT3+ew43yn4XY2bzOww4Jt4Nwtg/3jszYzr/iJ+bglxO87GK+V74fs1WccpeEuaMryMaRF/swq5Fl0LQghJU/jheFPLjfEK21QWV4RfQL0Xp9kPz++l5B4z2x3fhyvhx+HO5AJ3p+D7vmX8PI5fWDaJ38H362C8MpwEGNKB43lx+t/wO1s/4hcTrwFT490dw1tSJa3xkn20gMXrYnPx45iY3tLU9Mm++wo/R+4IYGZbW+y1Pc38jRuH4HfUXojbZRB+nk7uvPeN03bH8/iP8ee74xca/w8/pyfpLcMDDQfgZRDk+tnoid+t7IVflK6ElyXg55K0lix57ksUUX6FPF8SqNkJ3z7PEu/ax/zwFd68eBV8v06My07Kn9/I3dnM1wzf79/ij0qsEEJYPW6DJnEe6TK/jPh8Mv7cdLqifzm+DacDxTEfl+VNk76ASLci/DT+n1xATwb+HUJoEUJoFkJYCT9/pPNkEtzfAt83a+N3fn9P7tEf8AuiPeMybojDDsEv+r4l16onMRi/0UDwHv6T4+9EfB9PItc0uwm5croNLJlXY5DkQ7w8PAW/c78auWPvAHJ3be9h8UDpCngZNjdun+3wcucjcvn0A3I30ZJyNNE0bhvw8+s9LB6QHoGXFUmdsklcP/CyfZU4bDVyj56V4eflpCl6E7y8A398dys8z7XFg/XPxnUtW8qLuaICw0rj8J/wcvMY/MZdwI/tJKCQlFeP448RbxhC2C7OYwF+nkvOTzPiZ3Zcj/+Hl9kz8EDDLPy42iTmh3nkWnnMxPdPW3KP0iatp7qk0t0Z3xcD8brvrcD/FSjrwYNOLeNy74k3LTrgZf0LsV66cxz2DX6ePAWvizfFg4rrg78FIplpDK7Nx/PbS3jZtwneSuU1M+sYt13B3+OtnlaNyz8ebylQnkPMrKmZrRvTOrzANK8AZ5pbBd/2r8ZAyPt4i4yX8UDBmcSbp2a2QQjhE/x8u0uslybnIMxsvdQ5qEoUKGggQgjz8ajzoALjXojjPsQPsnEs3vyqkDfwg2TkUqRhHF4YvR2b35TLzGbgF3vrhhCm4QfnDnik8nP8TuHNeMT+T3igoAV+kuiOX6CtFTP8jnjl7Eb8hNYy/mYHvDA7DT9JLMBP1sld5C/wytBo/OI1eeSgkEvwpkQT8INyJEve4UzrY/5qmG/iumyABwbexi9kRgGTgj+e8UZcr6bxN03xbf9yiI9fBH8k4WTgLjObHNP8FR4pTewUf/8ZHjV9tZy0NcVP3Pvj2/dsvND5BW8yCH735SO8cN8XvwBP7rpsi58AK1vOmXgTqLF4JS65gH8C2DkOXzkOGxynuRu/6F0rpnNl8NfV4BdsyYn+OuA881fdfYrvj71YMnCyMn6COA+/iLkWby3wD3wfTCmQ7p/wk1lSoQQvtHfF74Klg23P4CfC/xSYTxIZPwwPtAAQQpiDn0SSi7Kf8ZPRZzGte+N5fRYeNf8Ar3zMxu+QTMPv3LXH98NT+MVLE3wfFuGVj4XkTtg/4ye5jvgdGuL2uisOG4pXDiEX2BiM54tOwGbxcRgAzGxvM0uCKKV4xRP8ImM+fvI7PA5LLsQMr2TthFdeCgUhf8ArvMV4hWQWXrEuwvPkzXhl8Hy8cng6uSaX18fl74cfyzvid1iSO7L/wsuCtnG7fI7vV+I23wg/Hmbj2z9pZXRq/D4fPzGvEKdLLjTBT/Rz8TKnS/y+Or4/5+KV/vZx2UmlqRm+z1bEAw2Jefj+fQo/htbF8wf4dm6D33nsBCy03Ku62pVTAfgIvxg8Fs/TyUXo7/BAzDMsXpbdggdlesbv3czsK/zxhpNS032Fn1M+wMve6eQeD1sZv2gYix/LT5F7NOZJcsGbX/F8ty65i8Y5+E11w7ddVzxAlp9fSmOw9yu8or5WbKraEq94WqwwGV7WbRbntQW+PY/Dg5M98XJldlyXI/GA0iz8rtyGLB487JJKwxxyF8C/4nfbkjtzY/GWOH+O224YufKuPHPw4/tRM3uCwo+bJWXI8cAg89ftXl/OtAcS7zKTK2Mvw8+TPfEge3N8Hw3C80Iyn6QV20t4PmliZi8QK8txPWfEaX/Cy5g18bvmn5Hr36YzXk4tjMsiBp2+TQWd8g2n/Ap5Ia/jd31DDGDOAubHY2MocH48NlrHNH2JX3x2jelejVxrDcjtw6H4/l0d6GRmm5rZbnHcJLw8glzQ8Gd8H7Yi19Eq+DY9Pn5fKa5bU/yYSN/h7hPX9zA8oJjep6fjx0lysbWHma0fL7z3p7D3yPWv0AM/h8/Gy9nkeHsb3++b4cFM8HLycvw47YgHAefirzz9nMWDpK/h+32FuKzH43Yx4P/FvHpPXO+hFM6ro/ByfTq5NzMk55tueHnYkcVvVsyK6b0BP+ab4GV6W7xMaIqXT93x8/UB5C7YwfPlHnhd4hs8kHYgi99E+1387SvkzldPxb8T8LJ6c/wYT/b1I/i23Bw/ln/At2cpcFGsv7XA9+VnMb8eQ6osr+LF3NvAinmtj17Fy7B25IJXSXn/PrlHJfeN44vj8tqa2bapYR3xshG8rGiB1/t+wOtOl+HXES3xPLwV8H68oE1uSHyKb+NZeP44C99HffC6+Fl4edQED+S0wh+XuTFuy4rW/4c4fXJu/xgvXwbgeWA0Xob8H57X1sHrWkfh++qVWG6esfhseRkP2HyH5/+X8f3UB6+X3FTB7z/C8/4XeJCzopuxU/D99AreSfkPBaYZkFrHt/G+Nr6K417BO4IdhwezNiFXl+ljZpPwY+3sOOx44IJY536ognQVFmq4sz19GtcHL2yX63VOGaR5O5biVUrxN99SSU/vdXk74hfOl9TU/JcjXUeQ6ygz4BeYk2NBuWfetL3wCsZk/O77FeReg9QTrxCNwSsel1awzEHkOs7ZgdhxVPw+Gtg4/n9h3O9f4ZWLJvgJdCG5zuJ6kut4p0OcNv3mkGSdppB7HOZycp3TrBjHN0unjdwjFbPw58rAT/aj8UrOi+R6JU+aiz5NrrO0cXgQoTV+0f0VHu2/G6/wdI5pfw2/2F4nb7vchV+o98XvNP2dXEX2Dbw10P/hlbbP8JPUj3Edn8Irhw/iFbf5+EXIteQ6vkvm+xK53tQvwVspnRd/k3RodjReUTgPr8SUxHWfGNP8dpxvCV5peDD+/TEu9+c4/nP8JLtZnG96P3QkdmIZvz8X03Y2uU5ik1YmZfhF4o9xu44h10/BrLjtn4zb8Pu43ebhlfjngGmp5SxKQwX5dTy5TkG3idt0LH4Hdy3yeuLPW6/D43o/Ebf3vfnHQPy+d1yXpDl5u7w0XB73z1v4xfGXeAUk6cn7/+EVxcfiJ1l+wCtbX8Z9mnRq+m9yHfL2jNPdglc6P4r77BtyHXYNwitxU/HK/P74hc8C/KLxIHIdcYW4L4rIvU2lDD8ePkutz49x/8yMaWyLB/mG4RdB7+IXh5fH3yfr9CJeBmwc98cvMV3v4vnla3K9ebeOv3mTXKd9i/YXXp4Mw/dxu/Kmq8bydtH884b3ZclXdKY7wryaXIfIi6WLAuVkKt9eR8XHXXp9j8OPnW/xPn8WS0PeOhwa90m6o8X38WNjPB7ImBL3xZQ4zSH4hdysmIbfyHWeOQovU9bDy5OFeN6agDdrX4hfOP1I7vGTgAezv42/fSYOPxIvB28i13Hbz3H7TCX3Vov87dMXL0d/Sm2fcfH3e5PrNHM2fvz0JFWPwY+xLngZ+XFcdvJYxAy8PCsj1+ndKHKPiD1Hrr+G5HxzGH7MfoNfRIY43OJvF8bxz8Xhz8T0jsUvLsew5OsRx6TywMdA79T++wm/8N2XXBDuDvw46hu/VymvUD359ey4vS5n8fz6IR5QO47c42NDyNUDVkttz8/wAP1acV7f4xeW5+FlS8GynNxbQXqm5rtp3C5jWbIzwxn4OWZS3GZl+MXk1/H/e+PvviaWg8BeqeNiIrny9Dc8OD2U3NvO+pJ7PeJeeBB+bJzPifH7RDw/vBG3yf/IdYb7Jh4EfzXZT3h+vQ+vz03H+zmrqP46jtz+H0w8f5RXfumz7J/k1Rkiyy02834E752/XmSsGI19FG/G91gF022NvwN6hpkdg9/R3LIm1rOmt6OZtcFP7HsG76yqTjDvE2EI/rqYYVmnpyrM7HR8P/XPOi1SvcxsTfzC7hsz2xS/yDwFr6jNMbOL8NeIJh2DbhBCGGlmnfEL6N+HED4ofwl1n5mNxytjk/OGX45X8K4q9DupH8zsTfyiZ2je8L74fj85i3SJJArVV2o7f9bHuklDlZ8f4vcx+A2eo+I0fVH5VW0KPeMistTM7E94c/+jk4tbM/t/5Dpxgdwzg9W1zDfJPf8I8HgI4aKlnM13eJOdivonAG96+ayZBTxyemQIIVRTGhYptB2XR3wUIe1BvAnjFdUZJCiwnH+EEAYuxe874Xd776kvJ+LYNLgLueZ8mNnD5DrPA3gnhHBCLSdNqkcb/Jhvg991ORVvhvh9fPZ7Pv6ml9n4nck1zWwhfufy2roWJDCz7fFOL9N+H/y5UllKde1Yj4GtIXmDzwkhFHosSiQzVc2rscx6luWoryxtuVegTvc2/jhNvamb1DcF8kNTcv0QJM7Bz7OL8kMqf/yPxfvnqGhZ+fkheYQy/ciqzospalEgIo3G8gY06gIzuw5vBpqYEELYYxnnVenFTnUHw5ZVY7rQ1UVozatqvq4r+b+xq43jvzqWsSznmHi870fs8A9/ZOGDpSnX63L5mHfOSvqr+C5+r3fn4KpYlv2RxT6M+XV1chfMM4AX0+ebrPJWXTsPNlYKFIiIiIiIiIjIInrrgYiIiIiIiIgsokCBiIiIiIiIiCyiQIGIiIjUWWbWqsCwHWKnkiIiIlIDFCgQERGRzJlZRzMrNrNBeaNezJuuG/5u9k3MbFj8fBs/w+LrG0VERGQ56PWIIiIiUhccDjQHisysM/BQHL5VfPsAwGXA7cBhIYSvgR0AzOxk/BW8g2o1xSIiIg2UAgUiIiJSF2wOtAK2AY4LIfQEf0Vh6v/DgSOAA8xscghhcEZpFRERadD06IGIiIjUBZ2Ai4DhwFQzezO2JGhvZsPNbDjQA3/P/DHAv83sxXLnJiIiIstMLQpERESkLjgE2ATYHrCkFQGAmfXF6yyPAW8BL4cQ5plZu9TvLzWz/sCtIYRHai3VIiIiDZACBSIiIlIXPADsDEwAHk/1SwDQETDgSGB0Ob+/Un0UiIiIVA8FCkRERKQuuAKYB1wVQnjezDYGPg0hvJxqUfAfYFugZ2apFBERaQTUR4GIiIhkLoTwbd6gO4Ad4/8tfJIwvXZTJSIi0jipRYGIiIjUOSGE2Wb2gpl9CrQFDk2PN7NngJXMbBj+WkViHwWjQwgn13qCRUREGhALIWSdBhEREZGCzMyCKisiIiK1SoECEREREREREVlEfRSIiIiIiIiIyCIKFIiIiIiIiIjIIgoUiIiIiIiIiMgiChSIiIiIiIiIyCIKFIiIiIiIiIjIIgoUiIiIiIiIiMgiChSIiIiIiIiIyCL/H+uT1cs8QE6xAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 1152x1152 with 3 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "\n",
    "plt.figure(figsize=(16,16))\n",
    "plt.rcParams['font.sans-serif']=['Microsoft YaHei']\n",
    "plt.subplot(3,1,1)\n",
    "plt.title('特征重要性')\n",
    "sns.barplot(data=fea_imp_df2.iloc[:10,:],x='norm_importance',y='feature')\n",
    " \n",
    "plt.subplot(3,1,2)\n",
    "plt.title('特征重要性累加图')\n",
    "plt.xlabel('特征个数')\n",
    "plt.ylabel('cum_importance')\n",
    "plt.plot(list(range(1, len(fea_names)+1)),fea_imp_df2['cum_importance'], 'r-')\n",
    " \n",
    "plt.subplot(3,1,3)\n",
    "plt.title('各个特征的归一化得分')\n",
    "plt.xlabel('特征')\n",
    "plt.ylabel('norm_importance')\n",
    "plt.plot(fea_imp_df2.feature,fea_imp_df2['norm_importance'], 'b*-')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "特征重要性为0的变量个数为 ：6\n",
      "6\n"
     ]
    }
   ],
   "source": [
    "zero_imp_col = list(fea_imp_df2[fea_imp_df2.fea_importance==0].feature)\n",
    "fea_imp_df2 = fea_imp_df2[~(fea_imp_df2.feature.isin(zero_imp_col))]\n",
    "print('特征重要性为0的变量个数为 ：{}'.format(len(zero_imp_col)))\n",
    "print(len(zero_imp_col))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "特征重要性比较弱的变量个数为：15\n",
      "15\n"
     ]
    }
   ],
   "source": [
    "low_imp_col = list(fea_imp_df2[fea_imp_df2.cum_importance>=0.8].feature)\n",
    "print('特征重要性比较弱的变量个数为：{}'.format(len(low_imp_col)))\n",
    "print(len(low_imp_col))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(87020, 35) (87020, 14)\n"
     ]
    }
   ],
   "source": [
    "# 删除特征重要性为0和比较弱的特征\n",
    "drop_imp_col = zero_imp_col+low_imp_col\n",
    "df_final = df_raw.drop(drop_imp_col,axis=1)\n",
    "print(df_raw.shape,df_final.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_final.to_csv('final.csv',index=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "from sklearn.model_selection import train_test_split\n",
    "seed = 11223\n",
    "\n",
    "df_final=pd.read_csv('final.csv')\n",
    "colst=df_final.columns.tolist()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import MinMaxScaler,StandardScaler\n",
    "mm = StandardScaler()\n",
    "target=df_final['Disbursed']\n",
    "df_final=mm.fit_transform(df_final.drop(columns=['Disbursed']))\n",
    "colst.remove('Disbursed')\n",
    "df_final=pd.DataFrame(df_final,columns=colst)\n",
    "df_final['Disbursed']=target"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>City</th>\n",
       "      <th>Monthly_Income</th>\n",
       "      <th>Existing_EMI</th>\n",
       "      <th>Salary_Account</th>\n",
       "      <th>Var5</th>\n",
       "      <th>Loan_Amount_Submitted</th>\n",
       "      <th>Processing_Fee</th>\n",
       "      <th>EMI_Loan_Submitted</th>\n",
       "      <th>Source</th>\n",
       "      <th>DOB_year</th>\n",
       "      <th>DOB_month</th>\n",
       "      <th>DOB_day</th>\n",
       "      <th>LCD_day</th>\n",
       "      <th>Disbursed</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.810576</td>\n",
       "      <td>-0.017842</td>\n",
       "      <td>-0.092885</td>\n",
       "      <td>0.792991</td>\n",
       "      <td>-0.629600</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>3.428508e-16</td>\n",
       "      <td>-0.526876</td>\n",
       "      <td>0.051774</td>\n",
       "      <td>1.495156</td>\n",
       "      <td>-0.375842</td>\n",
       "      <td>-0.853424</td>\n",
       "      <td>1.555166</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.119674</td>\n",
       "      <td>-0.010953</td>\n",
       "      <td>-0.092885</td>\n",
       "      <td>0.969351</td>\n",
       "      <td>1.822838</td>\n",
       "      <td>-0.815224</td>\n",
       "      <td>3.428508e-16</td>\n",
       "      <td>0.489816</td>\n",
       "      <td>0.051774</td>\n",
       "      <td>0.441032</td>\n",
       "      <td>1.945018</td>\n",
       "      <td>0.390095</td>\n",
       "      <td>1.409642</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>-1.739374</td>\n",
       "      <td>-0.016693</td>\n",
       "      <td>-0.092885</td>\n",
       "      <td>-0.635850</td>\n",
       "      <td>-0.629600</td>\n",
       "      <td>0.229878</td>\n",
       "      <td>3.428508e-16</td>\n",
       "      <td>-0.526876</td>\n",
       "      <td>1.349175</td>\n",
       "      <td>1.120237</td>\n",
       "      <td>1.945018</td>\n",
       "      <td>-0.707286</td>\n",
       "      <td>0.149302</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-1.656989</td>\n",
       "      <td>-0.010953</td>\n",
       "      <td>-0.092885</td>\n",
       "      <td>-0.635850</td>\n",
       "      <td>0.563373</td>\n",
       "      <td>2.194669</td>\n",
       "      <td>3.428508e-16</td>\n",
       "      <td>-0.526876</td>\n",
       "      <td>1.349175</td>\n",
       "      <td>0.004093</td>\n",
       "      <td>0.913070</td>\n",
       "      <td>-0.681115</td>\n",
       "      <td>0.109147</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.224401</td>\n",
       "      <td>0.018898</td>\n",
       "      <td>0.535355</td>\n",
       "      <td>0.792991</td>\n",
       "      <td>2.456351</td>\n",
       "      <td>0.438898</td>\n",
       "      <td>3.428508e-16</td>\n",
       "      <td>-0.526876</td>\n",
       "      <td>4.389386</td>\n",
       "      <td>0.463009</td>\n",
       "      <td>-1.553939</td>\n",
       "      <td>1.514176</td>\n",
       "      <td>0.381993</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>87015</th>\n",
       "      <td>-1.739374</td>\n",
       "      <td>0.005994</td>\n",
       "      <td>0.271494</td>\n",
       "      <td>0.969351</td>\n",
       "      <td>0.544911</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>3.428508e-16</td>\n",
       "      <td>-0.526876</td>\n",
       "      <td>0.051774</td>\n",
       "      <td>0.967043</td>\n",
       "      <td>0.913070</td>\n",
       "      <td>-0.481259</td>\n",
       "      <td>-0.744510</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>87016</th>\n",
       "      <td>-1.015301</td>\n",
       "      <td>-0.019679</td>\n",
       "      <td>-0.092885</td>\n",
       "      <td>-1.312568</td>\n",
       "      <td>-0.872348</td>\n",
       "      <td>-0.648007</td>\n",
       "      <td>-1.248334e-01</td>\n",
       "      <td>0.890134</td>\n",
       "      <td>0.051774</td>\n",
       "      <td>-1.082785</td>\n",
       "      <td>0.384253</td>\n",
       "      <td>1.488341</td>\n",
       "      <td>-0.744510</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>87017</th>\n",
       "      <td>0.224401</td>\n",
       "      <td>0.027164</td>\n",
       "      <td>-0.092885</td>\n",
       "      <td>-0.909777</td>\n",
       "      <td>0.085098</td>\n",
       "      <td>3.365183</td>\n",
       "      <td>3.428508e-16</td>\n",
       "      <td>-0.526876</td>\n",
       "      <td>0.051774</td>\n",
       "      <td>1.981077</td>\n",
       "      <td>1.694349</td>\n",
       "      <td>-0.541307</td>\n",
       "      <td>-0.744510</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>87018</th>\n",
       "      <td>0.224401</td>\n",
       "      <td>0.018406</td>\n",
       "      <td>0.250385</td>\n",
       "      <td>0.969351</td>\n",
       "      <td>2.118978</td>\n",
       "      <td>1.693021</td>\n",
       "      <td>3.428508e-16</td>\n",
       "      <td>-0.526876</td>\n",
       "      <td>0.051774</td>\n",
       "      <td>0.600948</td>\n",
       "      <td>-0.865415</td>\n",
       "      <td>-0.481259</td>\n",
       "      <td>-0.744510</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>87019</th>\n",
       "      <td>0.119674</td>\n",
       "      <td>-0.007600</td>\n",
       "      <td>-0.092885</td>\n",
       "      <td>-1.547683</td>\n",
       "      <td>1.055126</td>\n",
       "      <td>1.233176</td>\n",
       "      <td>-6.337407e-01</td>\n",
       "      <td>2.307188</td>\n",
       "      <td>0.051774</td>\n",
       "      <td>-0.561389</td>\n",
       "      <td>1.945018</td>\n",
       "      <td>0.437146</td>\n",
       "      <td>-0.744510</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>87020 rows × 14 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "           City  Monthly_Income  Existing_EMI  Salary_Account      Var5  \\\n",
       "0      0.810576       -0.017842     -0.092885        0.792991 -0.629600   \n",
       "1      0.119674       -0.010953     -0.092885        0.969351  1.822838   \n",
       "2     -1.739374       -0.016693     -0.092885       -0.635850 -0.629600   \n",
       "3     -1.656989       -0.010953     -0.092885       -0.635850  0.563373   \n",
       "4      0.224401        0.018898      0.535355        0.792991  2.456351   \n",
       "...         ...             ...           ...             ...       ...   \n",
       "87015 -1.739374        0.005994      0.271494        0.969351  0.544911   \n",
       "87016 -1.015301       -0.019679     -0.092885       -1.312568 -0.872348   \n",
       "87017  0.224401        0.027164     -0.092885       -0.909777  0.085098   \n",
       "87018  0.224401        0.018406      0.250385        0.969351  2.118978   \n",
       "87019  0.119674       -0.007600     -0.092885       -1.547683  1.055126   \n",
       "\n",
       "       Loan_Amount_Submitted  Processing_Fee  EMI_Loan_Submitted    Source  \\\n",
       "0                   0.000000    3.428508e-16           -0.526876  0.051774   \n",
       "1                  -0.815224    3.428508e-16            0.489816  0.051774   \n",
       "2                   0.229878    3.428508e-16           -0.526876  1.349175   \n",
       "3                   2.194669    3.428508e-16           -0.526876  1.349175   \n",
       "4                   0.438898    3.428508e-16           -0.526876  4.389386   \n",
       "...                      ...             ...                 ...       ...   \n",
       "87015               0.000000    3.428508e-16           -0.526876  0.051774   \n",
       "87016              -0.648007   -1.248334e-01            0.890134  0.051774   \n",
       "87017               3.365183    3.428508e-16           -0.526876  0.051774   \n",
       "87018               1.693021    3.428508e-16           -0.526876  0.051774   \n",
       "87019               1.233176   -6.337407e-01            2.307188  0.051774   \n",
       "\n",
       "       DOB_year  DOB_month   DOB_day   LCD_day  Disbursed  \n",
       "0      1.495156  -0.375842 -0.853424  1.555166          0  \n",
       "1      0.441032   1.945018  0.390095  1.409642          0  \n",
       "2      1.120237   1.945018 -0.707286  0.149302          0  \n",
       "3      0.004093   0.913070 -0.681115  0.109147          0  \n",
       "4      0.463009  -1.553939  1.514176  0.381993          0  \n",
       "...         ...        ...       ...       ...        ...  \n",
       "87015  0.967043   0.913070 -0.481259 -0.744510          0  \n",
       "87016 -1.082785   0.384253  1.488341 -0.744510          0  \n",
       "87017  1.981077   1.694349 -0.541307 -0.744510          0  \n",
       "87018  0.600948  -0.865415 -0.481259 -0.744510          0  \n",
       "87019 -0.561389   1.945018  0.437146 -0.744510          0  \n",
       "\n",
       "[87020 rows x 14 columns]"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_final"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = df_final.drop(columns=['Disbursed'])\n",
    "Y = df_final['Disbursed']\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, Y, test_size=0.20, random_state=seed)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-04T08:43:31.967511Z",
     "iopub.status.busy": "2021-09-04T08:43:31.967002Z",
     "iopub.status.idle": "2021-09-04T08:43:31.972111Z",
     "shell.execute_reply": "2021-09-04T08:43:31.971333Z",
     "shell.execute_reply.started": "2021-09-04T08:43:31.967480Z"
    },
    "id": "647849E3E9DB4D298AEEEC65413D559C",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "X_train_min=X_train[:5000]\n",
    "y_train_min=y_train[:5000]\n",
    "X_test_min=X_test[:800]\n",
    "y_test_min=y_test[:800]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SVC, time cost: 0.307805, Accuracy: 0.983750, Precision: 0.983750, AUC: 0.500000\n"
     ]
    }
   ],
   "source": [
    "from sklearn.svm import SVC\n",
    "from time import time\n",
    "from sklearn.metrics import accuracy_score,precision_score,roc_auc_score\n",
    "\n",
    "clf = SVC(kernel='rbf')\n",
    "start = time()\n",
    "clf.fit(X_train_min,y_train_min)\n",
    "end=time()\n",
    "y_pred = clf.predict(X_test_min)\n",
    "predictions = [value for value in y_pred]\n",
    "accuracy = accuracy_score(y_test_min, predictions)\n",
    "precision= precision_score(y_test_min, predictions,average='micro')\n",
    "auc=roc_auc_score(y_test_min, predictions)\n",
    "print(\"SVC, time cost: %f, Accuracy: %f, Precision: %f, AUC: %f\" % (end-start, accuracy,precision, auc))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {
    "id": "2E799D0FB88E43EF81C4EE092FF6522B",
    "jupyter": {},
    "notebookId": "613339d986080500170052d9",
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Utility function to report best scores\n",
    "def report(results, n_top=3):\n",
    "    for i in range(1, n_top + 1):\n",
    "        candidates = np.flatnonzero(results['rank_test_score'] == i)\n",
    "        for candidate in candidates:\n",
    "            print(\"Model with rank: {0}\".format(i))\n",
    "            print(\"Mean validation score: {0:.3f} (std: {1:.3f})\"\n",
    "                  .format(results['mean_test_score'][candidate],\n",
    "                          results['std_test_score'][candidate]))\n",
    "            print(\"Parameters: {0}\".format(results['params'][candidate]))\n",
    "            print(\"\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "86A40BB01AF14F3E8BF2EB260FB1D311",
    "jupyter": {},
    "scrolled": true,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### (1) 网格搜索"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-04T08:43:38.700314Z",
     "iopub.status.busy": "2021-09-04T08:43:38.699782Z"
    },
    "id": "3D04030DD09847939544530002A12213",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "GridSearchCV took 30.80 seconds for 8 candidate parameter settings.\n",
      "Model with rank: 1\n",
      "Mean validation score: 0.987 (std: 0.000)\n",
      "Parameters: {'C': 1, 'gamma': 0.001, 'kernel': 'rbf'}\n",
      "\n",
      "Model with rank: 1\n",
      "Mean validation score: 0.987 (std: 0.000)\n",
      "Parameters: {'C': 1, 'gamma': 0.0001, 'kernel': 'rbf'}\n",
      "\n",
      "Model with rank: 1\n",
      "Mean validation score: 0.987 (std: 0.000)\n",
      "Parameters: {'C': 10, 'gamma': 0.001, 'kernel': 'rbf'}\n",
      "\n",
      "Model with rank: 1\n",
      "Mean validation score: 0.987 (std: 0.000)\n",
      "Parameters: {'C': 10, 'gamma': 0.0001, 'kernel': 'rbf'}\n",
      "\n",
      "Model with rank: 1\n",
      "Mean validation score: 0.987 (std: 0.000)\n",
      "Parameters: {'C': 100, 'gamma': 0.0001, 'kernel': 'rbf'}\n",
      "\n",
      "Model with rank: 1\n",
      "Mean validation score: 0.987 (std: 0.000)\n",
      "Parameters: {'C': 1000, 'gamma': 0.0001, 'kernel': 'rbf'}\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import GridSearchCV\n",
    "from sklearn.metrics import classification_report\n",
    "from sklearn.svm import SVC\n",
    "from time import time\n",
    "\n",
    "tuned_parameters = [{'kernel': ['rbf'], 'gamma': [1e-3, 1e-4],'C': [1, 10, 100, 1000]}]\n",
    "                   # {'kernel': ['linear'], 'C': [1, 10, 100, 1000]}]\n",
    "\n",
    "clf = GridSearchCV(SVC(), tuned_parameters, scoring='accuracy' )\n",
    "start = time()\n",
    "clf.fit(X_train_min, y_train_min)\n",
    "\n",
    "print(\"GridSearchCV took %.2f seconds for %d candidate parameter settings.\"\n",
    "      % (time() - start, len(clf.cv_results_['params'])))\n",
    "report(clf.cv_results_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "6677C4BF63D84004AE5D96F076F0DD94",
    "jupyter": {},
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### (2) 随机搜索"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-04T08:14:02.531425Z",
     "iopub.status.busy": "2021-09-04T08:14:02.531027Z"
    },
    "id": "2AD0CCF64D804B649DCA43D33699C3D0",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RandomizedSearchCV took 47.87 seconds for 20 candidates parameter settings.\n",
      "Model with rank: 1\n",
      "Mean validation score: 0.986 (std: 0.001)\n",
      "Parameters: {'C': 549.44347858983, 'gamma': 0.0005538456592816083, 'kernel': 'rbf'}\n",
      "\n",
      "Model with rank: 1\n",
      "Mean validation score: 0.986 (std: 0.001)\n",
      "Parameters: {'C': 142.77074811363198, 'gamma': 0.002616280705371372, 'kernel': 'rbf'}\n",
      "\n",
      "Model with rank: 3\n",
      "Mean validation score: 0.986 (std: 0.001)\n",
      "Parameters: {'C': 2669.4178997523, 'gamma': 0.0036423448712689897, 'kernel': 'rbf'}\n",
      "\n",
      "Model with rank: 3\n",
      "Mean validation score: 0.986 (std: 0.001)\n",
      "Parameters: {'C': 2063.5670573584885, 'gamma': 0.004796267514603569, 'kernel': 'rbf'}\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from time import time\n",
    "import scipy.stats as stats\n",
    "#from scipy.stats import loguniform\n",
    "#from sklearn.utils.fixes import loguniform\n",
    "\n",
    "from sklearn.model_selection import GridSearchCV, RandomizedSearchCV\n",
    "from sklearn.svm import SVC\n",
    "\n",
    "# build a classifier\n",
    "clf = SVC()\n",
    "\n",
    "# specify parameters and distributions to sample from\n",
    "param_dist = {'C': stats.expon(scale=1000), \n",
    "              'gamma': stats.expon(scale=.1),\n",
    "              'kernel': ['rbf']} \n",
    "              #'class_weight':['balanced', None]}\n",
    "#{'average': [True, False],\n",
    "#              'l1_ratio': stats.uniform(0, 1),\n",
    "#              'alpha': loguniform(1e-4, 1e0)}\n",
    "\n",
    "# run randomized search\n",
    "n_iter_search = 20\n",
    "random_search = RandomizedSearchCV(clf, param_distributions=param_dist,\n",
    "                                   n_iter=n_iter_search)\n",
    "\n",
    "start = time()\n",
    "random_search.fit(X_train_min, y_train_min)\n",
    "print(\"RandomizedSearchCV took %.2f seconds for %d candidates\"\n",
    "      \" parameter settings.\" % ((time() - start), n_iter_search))\n",
    "report(random_search.cv_results_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "DD67EBBC7F074AC8B0AC2E33C77F90A1",
    "jupyter": {},
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### (3) 贝叶斯优化"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-09-04T05:27:19.749117Z",
     "iopub.status.busy": "2021-09-04T05:27:19.748742Z",
     "iopub.status.idle": "2021-09-04T05:42:44.429163Z",
     "shell.execute_reply": "2021-09-04T05:42:44.426496Z",
     "shell.execute_reply.started": "2021-09-04T05:27:19.749089Z"
    },
    "id": "D88DC5E21D1A4CCA85C75C0B76E33850",
    "jupyter": {},
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "|   iter    |  target   |     C     |   gamma   |\n",
      "-------------------------------------------------\n",
      "| \u001b[0m 1       \u001b[0m | \u001b[0m 0.9862  \u001b[0m | \u001b[0m 549.3   \u001b[0m | \u001b[0m 0.007437\u001b[0m |\n",
      "| \u001b[0m 2       \u001b[0m | \u001b[0m 0.986   \u001b[0m | \u001b[0m 603.2   \u001b[0m | \u001b[0m 0.005904\u001b[0m |\n",
      "| \u001b[0m 3       \u001b[0m | \u001b[0m 0.986   \u001b[0m | \u001b[0m 424.2   \u001b[0m | \u001b[0m 0.006813\u001b[0m |\n",
      "| \u001b[0m 4       \u001b[0m | \u001b[0m 0.9862  \u001b[0m | \u001b[0m 438.1   \u001b[0m | \u001b[0m 0.009026\u001b[0m |\n",
      "| \u001b[0m 5       \u001b[0m | \u001b[0m 0.986   \u001b[0m | \u001b[0m 963.7   \u001b[0m | \u001b[0m 0.004451\u001b[0m |\n",
      "| \u001b[0m 6       \u001b[0m | \u001b[0m 0.9862  \u001b[0m | \u001b[0m 10.13   \u001b[0m | \u001b[0m 0.003315\u001b[0m |\n",
      "| \u001b[0m 7       \u001b[0m | \u001b[0m 0.9862  \u001b[0m | \u001b[0m 10.13   \u001b[0m | \u001b[0m 0.006405\u001b[0m |\n",
      "| \u001b[95m 8       \u001b[0m | \u001b[95m 0.9864  \u001b[0m | \u001b[95m 445.5   \u001b[0m | \u001b[95m 0.001723\u001b[0m |\n",
      "| \u001b[0m 9       \u001b[0m | \u001b[0m 0.9864  \u001b[0m | \u001b[0m 451.1   \u001b[0m | \u001b[0m 0.004036\u001b[0m |\n",
      "| \u001b[0m 10      \u001b[0m | \u001b[0m 0.9862  \u001b[0m | \u001b[0m 467.4   \u001b[0m | \u001b[0m 0.009518\u001b[0m |\n",
      "| \u001b[0m 11      \u001b[0m | \u001b[0m 0.9864  \u001b[0m | \u001b[0m 38.85   \u001b[0m | \u001b[0m 0.005084\u001b[0m |\n",
      "| \u001b[0m 12      \u001b[0m | \u001b[0m 0.9864  \u001b[0m | \u001b[0m 51.4    \u001b[0m | \u001b[0m 0.002685\u001b[0m |\n",
      "| \u001b[0m 13      \u001b[0m | \u001b[0m 0.9864  \u001b[0m | \u001b[0m 68.54   \u001b[0m | \u001b[0m 0.005699\u001b[0m |\n",
      "| \u001b[0m 14      \u001b[0m | \u001b[0m 0.9864  \u001b[0m | \u001b[0m 86.43   \u001b[0m | \u001b[0m 0.003714\u001b[0m |\n",
      "| \u001b[0m 15      \u001b[0m | \u001b[0m 0.9864  \u001b[0m | \u001b[0m 106.0   \u001b[0m | \u001b[0m 0.004561\u001b[0m |\n",
      "| \u001b[0m 16      \u001b[0m | \u001b[0m 0.9864  \u001b[0m | \u001b[0m 126.6   \u001b[0m | \u001b[0m 0.002558\u001b[0m |\n",
      "| \u001b[0m 17      \u001b[0m | \u001b[0m 0.986   \u001b[0m | \u001b[0m 148.1   \u001b[0m | \u001b[0m 0.008918\u001b[0m |\n",
      "| \u001b[0m 18      \u001b[0m | \u001b[0m 0.986   \u001b[0m | \u001b[0m 783.0   \u001b[0m | \u001b[0m 0.008072\u001b[0m |\n",
      "| \u001b[0m 19      \u001b[0m | \u001b[0m 0.9864  \u001b[0m | \u001b[0m 287.4   \u001b[0m | \u001b[0m 0.001853\u001b[0m |\n",
      "| \u001b[0m 20      \u001b[0m | \u001b[0m 0.9864  \u001b[0m | \u001b[0m 306.5   \u001b[0m | \u001b[0m 0.003218\u001b[0m |\n",
      "| \u001b[0m 21      \u001b[0m | \u001b[0m 0.9864  \u001b[0m | \u001b[0m 265.2   \u001b[0m | \u001b[0m 0.0018  \u001b[0m |\n",
      "| \u001b[0m 22      \u001b[0m | \u001b[0m 0.9862  \u001b[0m | \u001b[0m 243.2   \u001b[0m | \u001b[0m 0.009399\u001b[0m |\n",
      "| \u001b[95m 23      \u001b[0m | \u001b[95m 0.9864  \u001b[0m | \u001b[95m 328.1   \u001b[0m | \u001b[95m 0.009488\u001b[0m |\n",
      "| \u001b[0m 24      \u001b[0m | \u001b[0m 0.9864  \u001b[0m | \u001b[0m 350.1   \u001b[0m | \u001b[0m 0.009343\u001b[0m |\n",
      "| \u001b[0m 25      \u001b[0m | \u001b[0m 0.9862  \u001b[0m | \u001b[0m 371.7   \u001b[0m | \u001b[0m 0.008752\u001b[0m |\n",
      "| \u001b[0m 26      \u001b[0m | \u001b[0m 0.9856  \u001b[0m | \u001b[0m 873.4   \u001b[0m | \u001b[0m 0.008788\u001b[0m |\n",
      "| \u001b[0m 27      \u001b[0m | \u001b[0m 0.986   \u001b[0m | \u001b[0m 693.2   \u001b[0m | \u001b[0m 0.006   \u001b[0m |\n",
      "| \u001b[0m 28      \u001b[0m | \u001b[0m 0.9864  \u001b[0m | \u001b[0m 511.2   \u001b[0m | \u001b[0m 0.002263\u001b[0m |\n",
      "| \u001b[0m 29      \u001b[0m | \u001b[0m 0.9862  \u001b[0m | \u001b[0m 201.1   \u001b[0m | \u001b[0m 0.009237\u001b[0m |\n",
      "| \u001b[0m 30      \u001b[0m | \u001b[0m 0.9856  \u001b[0m | \u001b[0m 648.2   \u001b[0m | \u001b[0m 0.009784\u001b[0m |\n",
      "=================================================\n",
      "Final result: {'target': 0.9864, 'params': {'C': 328.0526159202747, 'gamma': 0.009488026689120445}}\n",
      "time cost: 136.8041753768921\n"
     ]
    }
   ],
   "source": [
    "from sklearn.svm import SVC\n",
    "from time import time\n",
    "from bayes_opt import BayesianOptimization\n",
    "from sklearn.model_selection import cross_val_score\n",
    "\n",
    "def svc_cv(C,gamma, data, targets):\n",
    "    estimator = SVC(C=C,gamma=gamma)\n",
    "    cval = cross_val_score(estimator, data, targets, scoring='accuracy', cv=5)\n",
    "    return cval.mean()\n",
    "\n",
    "def optimize_svc(data, targets):\n",
    "    def svc_crossval(C, gamma):\n",
    "        #C = expC#10 ** expC\n",
    "        #expGamma = expGamma#10 ** expGamma\n",
    "        return svc_cv(C=C,gamma=gamma, data=data, targets=targets)\n",
    "\n",
    "    optimizer = BayesianOptimization(\n",
    "        f=svc_crossval,\n",
    "        pbounds={\"C\": (1, 1000),\"gamma\": (0.001, 0.01)},\n",
    "        random_state=0,\n",
    "        verbose=2\n",
    "    )\n",
    "    optimizer.maximize()\n",
    "\n",
    "    print(\"Final result:\", optimizer.max)\n",
    "start = time()\n",
    "optimize_svc(X_train_min, y_train_min)\n",
    "end = time()\n",
    "print('time cost:',end-start)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "B54AB1CCA3BA408A8048AB02E4F6C6F3",
    "jupyter": {},
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "1、网格搜索整体效率最低，因为受限精度，这里采用的步进较大，相对参数空间小，平均而言，30S/8参数空间=3.75 。  \n",
    "2、随机搜索整体效率高于网格搜索，因为是随机选取的，效率相当于，48s/20参数空间=2.4。  \n",
    "3、贝叶斯优化整体效率最高，136s/10000参数空间=0.0136。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
